October 2017 - ghc-tickets

[GHC] #9364: GHCi (or haskeline?) confused by non-single-width characters
by GHC 14 Dec '17

14 Dec '17

#9364: GHCi (or haskeline?) confused by non-single-width characters -------------------------------------+------------------------------------- Reporter: cheater | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: GHCi | Version: 7.8.3 Keywords: linux, terminal, | Operating System: colour, color, prompt | Unknown/Multiple Architecture: Unknown/Multiple | Type of failure: Other Difficulty: Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Hi, there's an issue in the GHCi prompt. To reproduce: 1. Do everything in Linux (I don't have other operating systems); I'm using a version of the gnome 2 gnome-terminal. 2. Use the following to colourise the prompt in GHCi: echo -e :set prompt '"\033[32;1m%s\033[34;1m>\033[0m "' >> ~/.ghc/ghci.conf 3. Import quite a few modules (say, 5 to 10 should be enough) 4. Give the prompt enough text that it wraps around. It's best if you start the text with -- so that it's a comment and doesn't produce any output. For example, say I entered "-- bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", I now see the following: *Types Control.Monad.Writer Control.Monad Data.Functor Control.Applicative Data.Monoid> -- bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb (do note that the first set of b's is located on the same line as the prompt, and the second is located under the prompt) 5. Press Enter. You should now see: *Types Control.Monad.Writer Control.Monad Data.Functor Control.Applicative Data.Monoid> -- bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb *Types Control.Monad.Writer Control.Monad Data.Functor Control.Applicative Data.Monoid> 6. Press Ctrl-P or the up arrow key to select the previous history item. You will now see the following: *Types Control.Monad.Writer Control.Monad Data.Functor Control.Applicative Data.Monoid> -- bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb *Types Control.Monad.Writer Control.Monad Data.Functor Control.Applicative Data.Monoid> -- bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb 7. Press Ctrl-A or the Home cursor key to go to the beginning of the text entry, and then type X. You will now see the last prompt to be: *Types Control.Monad.Writer Control.Monad Data.Functor Control.Applicative Data.Monoid> -- bbbbbbbbbbbbX-- bbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb 8. If you press Enter, you will find out that prompt was actually (without line breaks): X-- bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb I tried resizing the terminal window to 45 chars to make the lines above shorter; I imported Data.Monoid and Data.Functor in a new GHCi. The prompt contained garbage, most likely badly wrapped escape codes. See attachment. As both likely are a single bug in disguise, I include both reports in this ticket. Creating a test case for whether things actually work in a terminal might be difficult, but creating test for whether things get wrapped correctly (and not testing how things work out in a terminal) should be simple if annoying. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9364> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 7

[GHC] #14126: Two Failing Test Cases in HEAD
by GHC 12 Dec '17

12 Dec '17

#14126: Two Failing Test Cases in HEAD -------------------------------------+------------------------------------- Reporter: Shayan-Najd | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Build System | Version: 8.3 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- In a freshly cloned GHC code base (c6462ab0...), in my system,`make test TEST="T13780c T13822"` fails.That is while `./validate` passes. Specifically, I follow the following steps: (1) `git clone --recursive git://git.haskell.org/ghc.git` (2) `cd ghc` (3) `cp mk/build.mk.sample mk/build.mk` (4) set `BuildFlavour = devel2` and then (5) ./boot (6) ./configure (7) make -j4 (8) make test TEST="T13780c T13822" I build in a variant of the docker image for GHC development on Linux (64 bit), if it matters. As it seems, @alanz also gets the similar result on his system. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14126> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 5

[GHC] #13210: Can't run terminfo code in GHCi on Windows
by GHC 12 Dec '17

12 Dec '17

#13210: Can't run terminfo code in GHCi on Windows -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Runtime | Version: 8.0.2 System (Linker) | Keywords: | Operating System: Windows Architecture: | Type of failure: GHC rejects Unknown/Multiple | valid program Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- It's possible this is a duplicate of another runtime linker on Windows issue out there, but I'll post this just in case this really is a unique case. I was attempting to compile the `terminfo` library on Windows, and I actually made it surprisingly far--but I quickly stumbled when I actually tried to run some code in GHCi. First, make sure you have a native Windows version of `ncurses` installed. On MSYS2: {{{ $ pacman -S mingw-w64-x86_64-ncurses }}} And here's a stripped-down version of `terminfo`, which I'll use to demonstrate the bug: {{{#!hs module Main where import Control.Exception import Foreign import Foreign.C.String import Foreign.C.Types data TERMINAL newtype Terminal = Terminal (ForeignPtr TERMINAL) -- Use "unsafe" to make set_curterm faster since it's called quite a bit. foreign import ccall unsafe set_curterm :: Ptr TERMINAL -> IO (Ptr TERMINAL) foreign import ccall "&" del_curterm :: FunPtr (Ptr TERMINAL -> IO ()) foreign import ccall setupterm :: CString -> CInt -> Ptr CInt -> IO () setupTerm :: String -> IO Terminal setupTerm term = withCString term $ \c_term -> with 0 $ \ret_ptr -> do -- NOTE: I believe that for the way we use terminfo -- (i.e. custom output function) -- this parameter does not affect anything. let stdOutput = 1 -- Save the previous terminal to be restored after calling setupterm. old_term <- set_curterm nullPtr -- Call setupterm and check the return value. setupterm c_term stdOutput ret_ptr ret <- peek ret_ptr if (ret /=1) then throwIO $ SetupTermError $ "Couldn't look up terminfo entry " ++ show term else do cterm <- set_curterm old_term fmap Terminal $ newForeignPtr del_curterm cterm data SetupTermError = SetupTermError String instance Show SetupTermError where show (SetupTermError str) = "setupTerm: " ++ str instance Exception SetupTermError where main :: IO () main = do Terminal t <- setupTerm "xterm" print t }}} If you compile this code, it "works" (in the sense that it'll actually call the C code): {{{ $ ghc -lncursesw Terminfo.hs [1 of 1] Compiling Main ( Terminfo.hs, Terminfo.o ) Linking Terminfo.exe ... $ ./Terminfo Terminfo.exe: setupTerm: Couldn't look up terminfo entry "xterm" }}} But attempting to run the same code in GHCi is a disaster: {{{ $ runghc -lncursesw Terminfo.hs ghc.exe: C:/msys64/mingw64/lib/../lib/libncursesw.a: unknown symbol `nanosleep' ghc.exe: Could not on-demand load symbol '_nc_cookie_init' ghc.exe: C:/msys64/mingw64/lib/../lib/libncursesw.a: unknown symbol `_nc_cookie_init' ghc.exe: Could not on-demand load symbol '.refptr._nc_wacs' ghc.exe: C:/msys64/mingw64/lib/../lib/libncursesw.a: unknown symbol `.refptr._nc_wacs' ghc.exe: Could not on-demand load symbol '_nc_WIN_DRIVER' ghc.exe: C:/msys64/mingw64/lib/../lib/libncursesw.a: unknown symbol `_nc_WIN_DRIVER' ghc.exe: Could not on-demand load symbol '_nc_get_driver' ghc.exe: C:/msys64/mingw64/lib/../lib/libncursesw.a: unknown symbol `_nc_get_driver' ghc.exe: Could not on-demand load symbol '_nc_setupterm_ex' ghc.exe: C:/msys64/mingw64/lib/../lib/libncursesw.a: unknown symbol `_nc_setupterm_ex' ghc.exe: Could not on-demand load symbol 'newterm' ghc.exe: C:/msys64/mingw64/lib/../lib/libncursesw.a: unknown symbol `newterm' ghc.exe: Could not on-demand load symbol '.refptr._nc_globals' ghc.exe: C:/msys64/mingw64/lib/../lib/libncursesw.a: unknown symbol `.refptr._nc_globals' ghc.exe: Could not on-demand load symbol '_nc_err_abort' ghc.exe: C:/msys64/mingw64/lib/../lib/libncursesw.a: unknown symbol `_nc_err_abort' ghc.exe: Could not on-demand load symbol 'tparm' ghc.exe: C:/msys64/mingw64/lib/../lib/libncursesw.a: unknown symbol `tparm' ghc.exe: Could not on-demand load symbol '.refptr._nc_prescreen' ghc.exe: C:/msys64/mingw64/lib/../lib/libncursesw.a: unknown symbol `.refptr._nc_prescreen' ghc.exe: Could not on-demand load symbol '_nc_outch_sp' ghc.exe: C:/msys64/mingw64/lib/../lib/libncursesw.a: unknown symbol `_nc_outch_sp' ghc.exe: Could not on-demand load symbol '.refptr._nc_outch_sp' ghc.exe: C:/msys64/mingw64/lib/../lib/libncursesw.a: unknown symbol `.refptr._nc_outch_sp' ghc.exe: Could not on-demand load symbol '_nc_scrolln_sp' ghc.exe: C:/msys64/mingw64/lib/../lib/libncursesw.a: unknown symbol `_nc_scrolln_sp' ghc.exe: Could not on-demand load symbol '.refptr.SP' ghc.exe: C:/msys64/mingw64/lib/../lib/libncursesw.a: unknown symbol `.refptr.SP' ghc.exe: Could not on-demand load symbol '.refptr.cur_term' ghc.exe: C:/msys64/mingw64/lib/../lib/libncursesw.a: unknown symbol `.refptr.cur_term' ghc.exe: Could not on-demand load symbol '_nc_ospeed' ghc.exe: C:/msys64/mingw64/lib/../lib/libncursesw.a: unknown symbol `_nc_ospeed' ghc.exe: Could not on-demand load symbol 'set_curterm' Terminfo.hs: ByteCodeLink: can't find label During interactive linking, GHCi couldn't find the following symbol: set_curterm This may be due to you not asking GHCi to load extra object files, archives or DLLs needed by your current session. Restart GHCi, specifying the missing library using the -L/path/to/object/dir and -lmissinglibname flags, or simply by naming the relevant files on the GHCi command line. Alternatively, this link failure might indicate a bug in GHCi. If you suspect the latter, please send a bug report to: glasgow-haskell-bugs(a)haskell.org }}} Also reproducible on GHC HEAD. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13210> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 9

[GHC] #9669: Long compile time/high memory usage for modules with many deriving clauses
by GHC 12 Dec '17

12 Dec '17

#9669: Long compile time/high memory usage for modules with many deriving clauses -------------------------------------+------------------------------------- Reporter: snoyberg | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Keywords: | Operating System: Linux Architecture: x86_64 (amd64) | Type of failure: Compile- Difficulty: Unknown | time performance bug Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- I've seen many different people complain about this in various different contexts. One of the most common I personally see is when using Persistent with a large number of entities, which results in multi-minute build times for that single module. Usually, these reports have come from private repositories, which makes for a difficult repro. Also, the usage of Template Haskell in those modules tends to confuse the issue. So I'd like to report this issue from a completely separate project, with the guess that this represents an issue that is affecting many other users. Steps to reproduce: * `cabal unpack fpco-api-1.2.0.1` * `cd fpco-api-1.2.0.1` * `cabal install` * `ghc --make -isrc/library src/library/FP/API/Types.hs` * `touch src/library/FP/API/Types.hs` Then, to see the time and memory usage of compiling just the one module, run: * `time ghc -O2 --make -isrc/library src/library/FP/API/Types.hs +RTS -s` On my system (16GB RAM, quadcore i7), the results are: {{{ $ time ghc -O2 --make -isrc/library src/library/FP/API/Types.hs +RTS -s [3 of 3] Compiling FP.API.Types ( src/library/FP/API/Types.hs, src/library/FP/API/Types.o ) 51,846,533,568 bytes allocated in the heap 6,799,246,288 bytes copied during GC 389,297,584 bytes maximum residency (22 sample(s)) 11,330,472 bytes maximum slop 1041 MB total memory in use (0 MB lost due to fragmentation) Tot time (elapsed) Avg pause Max pause Gen 0 2041 colls, 0 par 9.19s 9.19s 0.0045s 0.0998s Gen 1 22 colls, 0 par 4.33s 4.33s 0.1969s 0.5436s TASKS: 4 (1 bound, 3 peak workers (3 total), using -N1) SPARKS: 0 (0 converted, 0 overflowed, 0 dud, 0 GC'd, 0 fizzled) INIT time 0.00s ( 0.00s elapsed) MUT time 34.21s ( 37.74s elapsed) GC time 13.52s ( 13.53s elapsed) EXIT time 0.07s ( 0.07s elapsed) Total time 47.81s ( 51.34s elapsed) Alloc rate 1,515,662,937 bytes per MUT second Productivity 71.7% of total user, 66.8% of total elapsed gc_alloc_block_sync: 0 whitehole_spin: 0 gen[0].sync: 0 gen[1].sync: 0 real 0m51.375s user 0m50.864s sys 0m0.456s }}} By contrast, with optimizations turned off: {{{ $ time ghc -O0 --make -isrc/library src/library/FP/API/Types.hs +RTS -s [3 of 3] Compiling FP.API.Types ( src/library/FP/API/Types.hs, src/library/FP/API/Types.o ) 12,767,593,936 bytes allocated in the heap 1,078,202,664 bytes copied during GC 179,551,768 bytes maximum residency (13 sample(s)) 6,684,544 bytes maximum slop 439 MB total memory in use (0 MB lost due to fragmentation) Tot time (elapsed) Avg pause Max pause Gen 0 623 colls, 0 par 1.24s 1.24s 0.0020s 0.0651s Gen 1 13 colls, 0 par 0.76s 0.76s 0.0586s 0.2397s TASKS: 4 (1 bound, 3 peak workers (3 total), using -N1) SPARKS: 0 (0 converted, 0 overflowed, 0 dud, 0 GC'd, 0 fizzled) INIT time 0.00s ( 0.00s elapsed) MUT time 8.28s ( 9.23s elapsed) GC time 2.00s ( 2.00s elapsed) EXIT time 0.05s ( 0.05s elapsed) Total time 10.34s ( 11.28s elapsed) Alloc rate 1,542,443,897 bytes per MUT second Productivity 80.6% of total user, 73.9% of total elapsed gc_alloc_block_sync: 0 whitehole_spin: 0 gen[0].sync: 0 gen[1].sync: 0 real 0m11.304s user 0m11.108s sys 0m0.172s }}} and -O1: {{{ $ time ghc -O1 --make -isrc/library src/library/FP/API/Types.hs +RTS -s [3 of 3] Compiling FP.API.Types ( src/library/FP/API/Types.hs, src/library/FP/API/Types.o ) 45,550,443,664 bytes allocated in the heap 5,721,700,512 bytes copied during GC 358,036,456 bytes maximum residency (21 sample(s)) 9,167,176 bytes maximum slop 906 MB total memory in use (0 MB lost due to fragmentation) Tot time (elapsed) Avg pause Max pause Gen 0 1642 colls, 0 par 7.70s 7.70s 0.0047s 0.1031s Gen 1 21 colls, 0 par 3.68s 3.69s 0.1756s 0.4968s TASKS: 4 (1 bound, 3 peak workers (3 total), using -N1) SPARKS: 0 (0 converted, 0 overflowed, 0 dud, 0 GC'd, 0 fizzled) INIT time 0.00s ( 0.00s elapsed) MUT time 30.21s ( 33.09s elapsed) GC time 11.38s ( 11.39s elapsed) EXIT time 0.07s ( 0.07s elapsed) Total time 41.68s ( 44.55s elapsed) Alloc rate 1,507,571,106 bytes per MUT second Productivity 72.7% of total user, 68.0% of total elapsed gc_alloc_block_sync: 0 whitehole_spin: 0 gen[0].sync: 0 gen[1].sync: 0 real 0m44.577s user 0m44.164s sys 0m0.368s }}} Note that this module is essentially just 900 lines of data declarations, with a large number of derived classes, and a few manually written helper functions and instances. This high memory usage hasn't just resulted in user complaints: automated Jenkins and Travis jobs will often times fail without optimizations disabled, which can be problematic for proper testing and production code. In the case of fpco-api, I've worked around this by adding `-O0` to the cabal file, but it would be much nicer to not have to do that. For some other examples of complaints along these lines: * https://groups.google.com/d/msg/yesodweb/MX1bnOFu8Hc/VLQXhnvpIkMJ * https://groups.google.com/d/msg/yesodweb/XPWixNjuOnM/FN26bmkudgwJ I believe there are a few other threads discussing this, if it would be helpful. I did my testing on GHC 7.8.3 64-bit, Ubuntu 12.04. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9669> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 19

[GHC] #13604: regression in ghc 8.2.1-rc1 (8.2.0.2017040)
by GHC 11 Dec '17

11 Dec '17

#13604: regression in ghc 8.2.1-rc1 (8.2.0.2017040) -------------------------------------+------------------------------------- Reporter: George | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: GHCi | Version: 8.0.1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: Runtime Unknown/Multiple | performance bug Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- In 8.2.1-rc1 loading a file compiled with -O2 into ghci results in ghci recompiling the file into interpreted byte code. In 8.0.2 it simply loads the compiled object file. 8.2.1 {{{ ghc -dynamic -O2 eh2.hs [1 of 1] Compiling Main ( eh2.hs, eh2.o ) Linking eh2 ... bash-3.2$ ghci -ignore-dot-ghci GHCi, version 8.2.0.20170404: http://www.haskell.org/ghc/ :? for help Prelude> :load eh2 [1 of 1] Compiling Main ( eh2.hs, interpreted ) [flags changed] Ok, modules loaded: Main. }}} 8.0.2 {{{ ghc --version The Glorious Glasgow Haskell Compilation System, version 8.0.2 bash-3.2$ pwd /Users/gcolpitts/haskell bash-3.2$ ghc -dynamic -O2 eh2.hs [1 of 1] Compiling Main ( eh2.hs, eh2.o ) Linking eh2 ... bash-3.2$ ghci -ignore-dot-ghci GHCi, version 8.0.2: http://www.haskell.org/ghc/ :? for help Prelude> :load eh2 Ok, modules loaded: Main (eh2.o). }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13604> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 52

[GHC] #14192: Change to 1TB VIRT allocation makes it impossible to core-dump Haskell programs
by GHC 08 Dec '17

08 Dec '17

#14192: Change to 1TB VIRT allocation makes it impossible to core-dump Haskell programs -------------------------------------+------------------------------------- Reporter: nh2 | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Runtime | Version: 8.0.2 System | Keywords: gdb, | Operating System: Unknown/Multiple debugging | Architecture: | Type of failure: Runtime Unknown/Multiple | performance bug Test Case: | Blocked By: Blocking: | Related Tickets: #9706 Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- GHC 8.0.2 on Linux changed the memory allocator to always allocate 1TB virtual memory on startup (#9706). I now have a production Haskell program running in a loop and would like to debug where it is stuck, on another machine, thus attaching with `gdb -p` and running `generate-core-file`. But core dumping takes forever, I Ctrl-C'd it when it reached 140 GB in size (my machine only has 64 GB RAM btw.); after the Ctrl-C the size of the core file on the file system was reported as `1.1T` (probably it's a sparse file now). Is there a workaround for this? For example, if I could dump only the resident or actually allocated pages, that would probably help. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14192> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 20

[GHC] #8457: -ffull-laziness does more harm than good
by GHC 08 Dec '17

08 Dec '17

#8457: -ffull-laziness does more harm than good ------------------------------------+------------------------------------- Reporter: errge | Owner: Type: bug | Status: new Priority: high | Milestone: 7.8.1 Component: Compiler | Version: 7.7 Keywords: | Operating System: Unknown/Multiple Architecture: Unknown/Multiple | Type of failure: None/Unknown Difficulty: Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | ------------------------------------+------------------------------------- In this bug report I'd like to argue that `-ffull-laziness` shouldn't be turned on automatically with either `-O` nor `-O2`, because it's dangerous and can cause serious memory leaks which are hard to debug or prevent. I'll also try to show that its optimization benefits are negligible. Actually, my benchmarks show that it's beneficial to turn it off even in the cases where we don't hit a space leak. We've met this issue last week, but it had been reported several times before: e.g. #917 and #5262. A typical example is the following: {{{ #!haskell main :: IO () main = task () >> task () task :: () -> IO () task () = printvalues [1..1000000 :: Int] printvalues :: [Int] -> IO () printvalues (x:xs) = print x >> printvalues xs printvalues [] = return () }}} We succeed with `-O0`, but fail with `-O`: {{{ errge@curry:~/tmp $ ~/tmp/ghc/inplace/bin/ghc-stage2 -v0 -O0 -fforce- recomp lazy && ./lazy +RTS -t >/dev/null <<ghc: 1620098744 bytes, 3117 GCs, 32265/42580 avg/max bytes residency (3 samples), 2M in use, 0.00 INIT (0.00 elapsed), 1.28 MUT (1.28 elapsed), 0.02 GC (0.02 elapsed) :ghc>> errge@curry:~/tmp $ ~/tmp/ghc/inplace/bin/ghc-stage2 -v0 -O -fforce-recomp lazy && ./lazy +RTS -t >/dev/null <<ghc: 1444098612 bytes, 2761 GCs, 3812497/13044272 avg/max bytes residency (7 samples), 28M in use, 0.00 INIT (0.00 elapsed), 1.02 MUT (1.03 elapsed), 0.12 GC (0.12 elapsed) :ghc>> }}} 28M? What the leak!? Well, it's `-ffull-laziness`: {{{ errge@curry:~/tmp $ ~/tmp/ghc/inplace/bin/ghc-stage2 -v0 -O -fno-full- laziness -fforce-recomp lazy && ./lazy +RTS -t >/dev/null <<ghc: 1484098612 bytes, 2835 GCs, 34812/42580 avg/max bytes residency (2 samples), 1M in use, 0.00 INIT (0.00 elapsed), 1.04 MUT (1.04 elapsed), 0.02 GC (0.02 elapsed) :ghc>> }}} We get constant space and the fastest run-time too, since we spare some cycles on GC. Note, that in this instance we are trying to explicity disable sharing by using `()` as a fake argument for the function. Also note, that this function may easily be a utility function in a larger code base or in a library, therefore it's impractical to say that you shouldn't use it twice "too close together". Quoting from the GHC user guide: {{{ -O2: Means: “Apply every non-dangerous optimisation, even if it means significantly longer compile times.” The avoided “dangerous” optimisations are those that can make runtime or space worse if you're unlucky. They are normally turned on or off individually. At the moment, -O2 is unlikely to produce better code than -O. }}} This seems to be false at the moment. We decided to make a broader investigation into this issue and wanted to know if we can disable this optimization without too much pain. Came up with this benchmark plan: - let's benchmark GHC, - compile all stages with -O, but hack the stage1 compiler to emit `-t` statistics for every file compiled, - gather these statistics while compiling the libraries and the stage2 compiler. On the second run we compile the stage1 compiler with `-O -fno-full-laziness`, but leave everything else unchanged in the environment. When we have both results of the compilation of ~1600 files, we match them up and compute the (logarithmic) ratio of CPU and memory difference between compilations, the final results for our benchmark. The results and the raw data can be found at https://github.com/errge/notlazy. The overall compilation time dropped from 26:20 to 25:12, which is a 4% improvement. Investigating the full matching shows that this overall result is from small improvements all around the place. The results plotted: - https://github.com/errge/notlazy/blob/master/cpu.png - https://github.com/errge/notlazy/blob/master/mem.png The graphs show the logarithmic (100*log_10(new/orig)) ratio of change in cpu and memory consumption. Therefore negative results mean that the new compilation method is faster. As can be seen on the CPU graph, in most of the cases the difference is negligible (actually smaller than what can be measured on small files, this is why we have the spike at 0). In overall we see a small improvement in CPU, and there are some outliers in both directions, but there are more drastic improvement cases than drastic regressions. On the memory graph the situation is much more close to zero. There is one big positive memory outlier: `DsListComp.lhs`. It uses 69M originally and now uses 103M. But compiles in 2 seconds both ways and there are files in the source tree which requires 400M to compile, so this is not an issue. After all this, I'd like to hear other opinions about just disabling this optimization in `-O` and `-O2` and leaving it as an option that can be turned on when needed, my reasons once more: - it's unsafe, - it's hard to debug when you hit its issues, - the optimization doesn't seem to be very productive, - it's always easy to force sharing, but it's not easy to force copying. Apparently a Haskell programmer should be lazy, but never fully lazy. Research done by Gergely Risko <errge> and Mihaly Barasz <klao>, confirmed on two different machines with no other running processes. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/8457> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 14

[GHC] #10535: Float out causes major space leak
by GHC 08 Dec '17

08 Dec '17

#10535: Float out causes major space leak -------------------------------------+------------------------------------- Reporter: AlexET | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Blocked By: Test Case: | Related Tickets: Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- This issue originally was on haskell-cafe https://mail.haskell.org/pipermail/haskell-cafe/2015-June/120113.html The following code has a space leak {{{ #!hs numbers = [1..200] replicateM' :: Monad m => Int -> m a -> m [a] replicateM' 0 xs = return [] replicateM' n xs = do a <- xs b <- replicateM' (n-1) xs return (a:b) test :: [[Int]] test = replicateM' 4 numbers main = print test }}} The recursive call in replicateM' gets floated out. This then causes a very large (200^3 elements) list to be kept in memory instead of being lazily produced and consumed each time. A similar thing happens to the replicateM in Control.Monad but it is harder to spot as a lot of specialisation and fusion goes on before the float-out happens This is similar to #7367 but in this case we have a huge space-leak instead of just adding allocations. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10535> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 7

[GHC] #14382: The 'impossible' happened whilst installing gi-gtk via cabal
by GHC 02 Dec '17

02 Dec '17

#14382: The 'impossible' happened whilst installing gi-gtk via cabal --------------------------------------+--------------------------------- Reporter: maartenjacobs | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.2.1 Keywords: gtk, pango | Operating System: Linux Architecture: x86_64 (amd64) | Type of failure: None/Unknown Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: --------------------------------------+--------------------------------- I've been trying to install gi-gtk using "cabal install gi-gtk". In the process I upgraded to the most recent stable GHC and cabal. I've also installed gtk3 libraries which I thought were necessary. {{{ ... [91 of 95] Compiling GI.Pango.Objects.Layout ( GI/Pango/Objects/Layout.hs, dist/build/GI/Pango/Objects/Layout.o ) ghc: panic! (the 'impossible' happened) (GHC version 8.2.1 for x86_64-unknown-linux): tcIfaceGlobal (local): not found You are in a maze of twisty little passages, all alike. While forcing the thunk for TyThing Layout which was lazily initialized by initIfaceCheck typecheckLoop, I tried to tie the knot, but I couldn't find Layout in the current type environment. If you are developing GHC, please read Note [Tying the knot] and Note [Type-checking inside the knot]. Consider rebuilding GHC with profiling for a better stack trace. Contents of current type environment: [] Call stack: CallStack (from HasCallStack): prettyCurrentCallStack, called at compiler/utils/Outputable.hs:1133:58 in ghc:Outputable callStackDoc, called at compiler/utils/Outputable.hs:1137:37 in ghc:Outputable pprPanic, called at compiler/iface/TcIface.hs:1696:23 in ghc:TcIface Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug Failed to install gi-pango-1.0.15 cabal: Error: some packages failed to install: gi-gdk-3.0.14-INozoUgbf2HFkX3VeIwKfl depends on gi-gdk-3.0.14 which failed to install. gi-gtk-3.0.17-1AObKz0Ppj5GXVupmEC7Yc depends on gi-gtk-3.0.17 which failed to install. gi-pango-1.0.15-E4HLwHGC2n62ObUIQeuwp8 failed during the building phase. The exception was: ExitFailure 1 }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14382> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 12

[GHC] #14254: The Binary instance for TypeRep smells a bit expensive
by GHC 01 Dec '17

01 Dec '17

#14254: The Binary instance for TypeRep smells a bit expensive -------------------------------------+------------------------------------- Reporter: dfeuer | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.2.1 Keywords: Typeable | Operating System: Unknown/Multiple Architecture: | Type of failure: Compile-time Unknown/Multiple | performance bug Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- In particular, `get` uses `getSomeTypeRep`. `getSomeTypeRep`, in turn, calls `typeRepKind` through its recursion. But `typeRepKind` is itself recursive, fully inspecting the spine of its argument. That smells quadratic to me. The solution, I believe, is to change the type of `getSomeTypeRep` to `BinHandle -> IO (SomeTypeRep, TypeRep Type)`. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14254> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 19