November 2017 - ghc-tickets

[GHC] #14192: Change to 1TB VIRT allocation makes it impossible to core-dump Haskell programs
by GHC 08 Dec '17

08 Dec '17

#14192: Change to 1TB VIRT allocation makes it impossible to core-dump Haskell programs -------------------------------------+------------------------------------- Reporter: nh2 | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Runtime | Version: 8.0.2 System | Keywords: gdb, | Operating System: Unknown/Multiple debugging | Architecture: | Type of failure: Runtime Unknown/Multiple | performance bug Test Case: | Blocked By: Blocking: | Related Tickets: #9706 Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- GHC 8.0.2 on Linux changed the memory allocator to always allocate 1TB virtual memory on startup (#9706). I now have a production Haskell program running in a loop and would like to debug where it is stuck, on another machine, thus attaching with `gdb -p` and running `generate-core-file`. But core dumping takes forever, I Ctrl-C'd it when it reached 140 GB in size (my machine only has 64 GB RAM btw.); after the Ctrl-C the size of the core file on the file system was reported as `1.1T` (probably it's a sparse file now). Is there a workaround for this? For example, if I could dump only the resident or actually allocated pages, that would probably help. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14192> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 20

[GHC] #8457: -ffull-laziness does more harm than good
by GHC 08 Dec '17

08 Dec '17

#8457: -ffull-laziness does more harm than good ------------------------------------+------------------------------------- Reporter: errge | Owner: Type: bug | Status: new Priority: high | Milestone: 7.8.1 Component: Compiler | Version: 7.7 Keywords: | Operating System: Unknown/Multiple Architecture: Unknown/Multiple | Type of failure: None/Unknown Difficulty: Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | ------------------------------------+------------------------------------- In this bug report I'd like to argue that `-ffull-laziness` shouldn't be turned on automatically with either `-O` nor `-O2`, because it's dangerous and can cause serious memory leaks which are hard to debug or prevent. I'll also try to show that its optimization benefits are negligible. Actually, my benchmarks show that it's beneficial to turn it off even in the cases where we don't hit a space leak. We've met this issue last week, but it had been reported several times before: e.g. #917 and #5262. A typical example is the following: {{{ #!haskell main :: IO () main = task () >> task () task :: () -> IO () task () = printvalues [1..1000000 :: Int] printvalues :: [Int] -> IO () printvalues (x:xs) = print x >> printvalues xs printvalues [] = return () }}} We succeed with `-O0`, but fail with `-O`: {{{ errge@curry:~/tmp $ ~/tmp/ghc/inplace/bin/ghc-stage2 -v0 -O0 -fforce- recomp lazy && ./lazy +RTS -t >/dev/null <<ghc: 1620098744 bytes, 3117 GCs, 32265/42580 avg/max bytes residency (3 samples), 2M in use, 0.00 INIT (0.00 elapsed), 1.28 MUT (1.28 elapsed), 0.02 GC (0.02 elapsed) :ghc>> errge@curry:~/tmp $ ~/tmp/ghc/inplace/bin/ghc-stage2 -v0 -O -fforce-recomp lazy && ./lazy +RTS -t >/dev/null <<ghc: 1444098612 bytes, 2761 GCs, 3812497/13044272 avg/max bytes residency (7 samples), 28M in use, 0.00 INIT (0.00 elapsed), 1.02 MUT (1.03 elapsed), 0.12 GC (0.12 elapsed) :ghc>> }}} 28M? What the leak!? Well, it's `-ffull-laziness`: {{{ errge@curry:~/tmp $ ~/tmp/ghc/inplace/bin/ghc-stage2 -v0 -O -fno-full- laziness -fforce-recomp lazy && ./lazy +RTS -t >/dev/null <<ghc: 1484098612 bytes, 2835 GCs, 34812/42580 avg/max bytes residency (2 samples), 1M in use, 0.00 INIT (0.00 elapsed), 1.04 MUT (1.04 elapsed), 0.02 GC (0.02 elapsed) :ghc>> }}} We get constant space and the fastest run-time too, since we spare some cycles on GC. Note, that in this instance we are trying to explicity disable sharing by using `()` as a fake argument for the function. Also note, that this function may easily be a utility function in a larger code base or in a library, therefore it's impractical to say that you shouldn't use it twice "too close together". Quoting from the GHC user guide: {{{ -O2: Means: “Apply every non-dangerous optimisation, even if it means significantly longer compile times.” The avoided “dangerous” optimisations are those that can make runtime or space worse if you're unlucky. They are normally turned on or off individually. At the moment, -O2 is unlikely to produce better code than -O. }}} This seems to be false at the moment. We decided to make a broader investigation into this issue and wanted to know if we can disable this optimization without too much pain. Came up with this benchmark plan: - let's benchmark GHC, - compile all stages with -O, but hack the stage1 compiler to emit `-t` statistics for every file compiled, - gather these statistics while compiling the libraries and the stage2 compiler. On the second run we compile the stage1 compiler with `-O -fno-full-laziness`, but leave everything else unchanged in the environment. When we have both results of the compilation of ~1600 files, we match them up and compute the (logarithmic) ratio of CPU and memory difference between compilations, the final results for our benchmark. The results and the raw data can be found at https://github.com/errge/notlazy. The overall compilation time dropped from 26:20 to 25:12, which is a 4% improvement. Investigating the full matching shows that this overall result is from small improvements all around the place. The results plotted: - https://github.com/errge/notlazy/blob/master/cpu.png - https://github.com/errge/notlazy/blob/master/mem.png The graphs show the logarithmic (100*log_10(new/orig)) ratio of change in cpu and memory consumption. Therefore negative results mean that the new compilation method is faster. As can be seen on the CPU graph, in most of the cases the difference is negligible (actually smaller than what can be measured on small files, this is why we have the spike at 0). In overall we see a small improvement in CPU, and there are some outliers in both directions, but there are more drastic improvement cases than drastic regressions. On the memory graph the situation is much more close to zero. There is one big positive memory outlier: `DsListComp.lhs`. It uses 69M originally and now uses 103M. But compiles in 2 seconds both ways and there are files in the source tree which requires 400M to compile, so this is not an issue. After all this, I'd like to hear other opinions about just disabling this optimization in `-O` and `-O2` and leaving it as an option that can be turned on when needed, my reasons once more: - it's unsafe, - it's hard to debug when you hit its issues, - the optimization doesn't seem to be very productive, - it's always easy to force sharing, but it's not easy to force copying. Apparently a Haskell programmer should be lazy, but never fully lazy. Research done by Gergely Risko <errge> and Mihaly Barasz <klao>, confirmed on two different machines with no other running processes. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/8457> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 14

[GHC] #10535: Float out causes major space leak
by GHC 08 Dec '17

08 Dec '17

#10535: Float out causes major space leak -------------------------------------+------------------------------------- Reporter: AlexET | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Blocked By: Test Case: | Related Tickets: Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- This issue originally was on haskell-cafe https://mail.haskell.org/pipermail/haskell-cafe/2015-June/120113.html The following code has a space leak {{{ #!hs numbers = [1..200] replicateM' :: Monad m => Int -> m a -> m [a] replicateM' 0 xs = return [] replicateM' n xs = do a <- xs b <- replicateM' (n-1) xs return (a:b) test :: [[Int]] test = replicateM' 4 numbers main = print test }}} The recursive call in replicateM' gets floated out. This then causes a very large (200^3 elements) list to be kept in memory instead of being lazily produced and consumed each time. A similar thing happens to the replicateM in Control.Monad but it is harder to spot as a lot of specialisation and fusion goes on before the float-out happens This is similar to #7367 but in this case we have a huge space-leak instead of just adding allocations. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10535> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 7

[GHC] #14524: Use sortOn instead of sortWith
by GHC 04 Dec '17

04 Dec '17

#14524: Use sortOn instead of sortWith -------------------------------------+------------------------------------- Reporter: AndreasK | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.2.1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: Other Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- `sortWith`provides the same functionality as `sortOn` from `Data.List`. As sortOn is already part of Prelude it makes sense for the GHC source to use it as well. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14524> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 3

[GHC] #14382: The 'impossible' happened whilst installing gi-gtk via cabal
by GHC 02 Dec '17

02 Dec '17

#14382: The 'impossible' happened whilst installing gi-gtk via cabal --------------------------------------+--------------------------------- Reporter: maartenjacobs | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.2.1 Keywords: gtk, pango | Operating System: Linux Architecture: x86_64 (amd64) | Type of failure: None/Unknown Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: --------------------------------------+--------------------------------- I've been trying to install gi-gtk using "cabal install gi-gtk". In the process I upgraded to the most recent stable GHC and cabal. I've also installed gtk3 libraries which I thought were necessary. {{{ ... [91 of 95] Compiling GI.Pango.Objects.Layout ( GI/Pango/Objects/Layout.hs, dist/build/GI/Pango/Objects/Layout.o ) ghc: panic! (the 'impossible' happened) (GHC version 8.2.1 for x86_64-unknown-linux): tcIfaceGlobal (local): not found You are in a maze of twisty little passages, all alike. While forcing the thunk for TyThing Layout which was lazily initialized by initIfaceCheck typecheckLoop, I tried to tie the knot, but I couldn't find Layout in the current type environment. If you are developing GHC, please read Note [Tying the knot] and Note [Type-checking inside the knot]. Consider rebuilding GHC with profiling for a better stack trace. Contents of current type environment: [] Call stack: CallStack (from HasCallStack): prettyCurrentCallStack, called at compiler/utils/Outputable.hs:1133:58 in ghc:Outputable callStackDoc, called at compiler/utils/Outputable.hs:1137:37 in ghc:Outputable pprPanic, called at compiler/iface/TcIface.hs:1696:23 in ghc:TcIface Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug Failed to install gi-pango-1.0.15 cabal: Error: some packages failed to install: gi-gdk-3.0.14-INozoUgbf2HFkX3VeIwKfl depends on gi-gdk-3.0.14 which failed to install. gi-gtk-3.0.17-1AObKz0Ppj5GXVupmEC7Yc depends on gi-gtk-3.0.17 which failed to install. gi-pango-1.0.15-E4HLwHGC2n62ObUIQeuwp8 failed during the building phase. The exception was: ExitFailure 1 }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14382> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 12

[GHC] #14254: The Binary instance for TypeRep smells a bit expensive
by GHC 01 Dec '17

01 Dec '17

#14254: The Binary instance for TypeRep smells a bit expensive -------------------------------------+------------------------------------- Reporter: dfeuer | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.2.1 Keywords: Typeable | Operating System: Unknown/Multiple Architecture: | Type of failure: Compile-time Unknown/Multiple | performance bug Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- In particular, `get` uses `getSomeTypeRep`. `getSomeTypeRep`, in turn, calls `typeRepKind` through its recursion. But `typeRepKind` is itself recursive, fully inspecting the spine of its argument. That smells quadratic to me. The solution, I believe, is to change the type of `getSomeTypeRep` to `BinHandle -> IO (SomeTypeRep, TypeRep Type)`. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14254> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 19

[GHC] #14490: TTG Snags
by GHC 01 Dec '17

01 Dec '17

#14490: TTG Snags -------------------------------------+------------------------------------- Reporter: alanz | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.3 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: #14482 Differential Rev(s): | Wiki Page: | ImplementingTreesThatGrow -------------------------------------+------------------------------------- == Context ImplementingTreesThatGrow The intention is to get the first-pass implementation of Trees that Grow into GHC 8.4.1. So far the following commits are on master - https://phabricator.haskell.org/rGHC438dd1cbba13d35f3452b4dcef3f94ce9a216905 - https://phabricator.haskell.org/rGHCe3ec2e7ae94524ebd111963faf34b84d942265b4 - https://phabricator.haskell.org/rGHC47ad6578ea460999b53eb4293c3a3b3017a56d65 Unfortunately we have picked up compile-time performance regressions in compiling GHC itself, with these patches. The perf.haskell.org results are - https://perf.haskell.org/ghc/#revision/438dd1cbba13d35f3452b4dcef3f94ce9a21… - https://perf.haskell.org/ghc/#revision/e3ec2e7ae94524ebd111963faf34b84d9422… - https://perf.haskell.org/ghc/#revision/47ad6578ea460999b53eb4293c3a3b3017a5… Summary: each of the three patches caused GHC validate time to worsen by another 5% or so, cumulatively pushing compile time from 2011 to 2400 seconds. It seems the major culprit is deriving the `Data` instances, probably related to the complex constraint sets required to ensure that all the extension points have `Data` instances too. Some discussion around this is captured in ImplementingTreesThatGrow/Instances == Things tried so far PLAN E (ImplementingTreesThatGrow/Instances#PLANE), in https://github.com/ghc/ghc/tree/wip/ttg5-data-one-file-2017-11-17. This puts all the `Data` derivations for the hsSyn AST into a single file, `hsSyn/HsInstances.hs`. This causes compile-time memory usage to spike up to around 7GB when compiling this file. Gipedia reports a compilation time of 2468 secs for this version, which perhaps indicates that the instance derivation is not the main culprit. The module itself only takes in the order of 45secs to compile, on `home .smart-cactus.org` (http://lpaste.net/7779809210564345856) Splitting the `HsInstances.hs` file in 2, as per https://github.com/ghc/ghc/tree/wip/ttg5-data-2017-11-17 reduces the memory requirement for each, but shows up #14482, where there is a linker failure due to incorrectly processing boot files. == Next Steps I guess the first thing to do is to try to isolate what precisely is causing the slowdown. It may not be instance generation at all. Also, it is probably better to revert the three patches from master, when cutting the 8.4 initial freeze. Open to suggestions ... -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14490> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 48

[GHC] #14539: untouchable type inside the constraints
by GHC 01 Dec '17

01 Dec '17

#14539: untouchable type inside the constraints -------------------------------------+------------------------------------- Reporter: Lemming | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.2.2 (Type checker) | Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: GHC rejects Unknown/Multiple | valid program Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- The following code is taken and simplified from the test-suite of `accelerate-fourier`: {{{ {-# LANGUAGE TypeFamilies #-} {-# LANGUAGE Rank2Types #-} module UntouchableType (tests) where import Test.QuickCheck (Testable, Arbitrary, arbitrary, quickCheck, ) newtype Sign a = Sign a deriving (Show) instance Arbitrary (Sign a) where arbitrary = undefined quickCheckWithSign :: (Testable prop) => (Sign Double -> prop) -> IO () quickCheckWithSign = quickCheck data Array sh a = Array type family FullShape sh :: * data SubTransform a = SubTransform (forall sh. (FullShape sh ~ sh) => Array sh a) transform2d :: SubTransform Double -> Bool transform2d = undefined transformChirp2 :: Sign a -> Array sh a transformChirp2 = undefined tests :: IO () tests = quickCheck $ \sign -> transform2d (SubTransform (transformChirp2 (sign::Sign Double))) }}} GHC-8.2.2 says about it: {{{ [1 of 1] Compiling UntouchableType ( UntouchableType.hs, interpreted ) UntouchableType.hs:34:4: error: • Ambiguous type variable ‘p0’ arising from a use of ‘quickCheck’ prevents the constraint ‘(Arbitrary p0)’ from being solved. Probable fix: use a type annotation to specify what ‘p0’ should be. These potential instances exist: instance (Arbitrary a, Arbitrary b) => Arbitrary (Either a b) -- Defined in ‘Test.QuickCheck.Arbitrary’ instance Arbitrary Ordering -- Defined in ‘Test.QuickCheck.Arbitrary’ instance Arbitrary Integer -- Defined in ‘Test.QuickCheck.Arbitrary’ ...plus 20 others ...plus 62 instances involving out-of-scope types (use -fprint-potential-instances to see them all) • In the expression: quickCheck $ \ sign -> transform2d (SubTransform (transformChirp2 (sign :: Sign Double))) In an equation for ‘tests’: tests = quickCheck $ \ sign -> transform2d (SubTransform (transformChirp2 (sign :: Sign Double))) | 34 | quickCheck $ \sign -> | ^^^^^^^^^^^^^^^^^^^^^... UntouchableType.hs:35:51: error: • Couldn't match expected type ‘Sign Double’ with actual type ‘p0’ ‘p0’ is untouchable inside the constraints: FullShape sh ~ sh bound by a type expected by the context: forall sh. FullShape sh ~ sh => Array sh Double at UntouchableType.hs:35:20-69 • In the first argument of ‘transformChirp2’, namely ‘(sign :: Sign Double)’ In the first argument of ‘SubTransform’, namely ‘(transformChirp2 (sign :: Sign Double))’ In the first argument of ‘transform2d’, namely ‘(SubTransform (transformChirp2 (sign :: Sign Double)))’ • Relevant bindings include sign :: p0 (bound at UntouchableType.hs:34:18) | 35 | transform2d (SubTransform (transformChirp2 (sign::Sign Double))) | ^^^^ Failed, no modules loaded. }}} I have tested GHC versions back to GHC-7.4.2, all of them report essentially the same type error. I do not really understand the type error message, but here are my observations that I find strange: The type annotation `sign :: Sign Double` does not prevent the type error, but replacing `quickCheck` by `quickCheckWithSign` does. Replacing the constraint `FullShape sh ~ sh` by, e.g. `Show sh`, let the error disappear. Generalizing `transform2d` to `SubTransform a` causes another type error although I expected that `SubTransform Double` can be infered from `Sign Double`. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14539> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 2

[GHC] #14531: tcIfaceGlobal (local): not found
by GHC 01 Dec '17

01 Dec '17

#14531: tcIfaceGlobal (local): not found --------------------------------------+--------------------------------- Reporter: bigos | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.2.1 Keywords: Windows Msys2 | Operating System: Windows Architecture: x86_64 (amd64) | Type of failure: None/Unknown Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: --------------------------------------+--------------------------------- I have installed Msys2 on Windows. In Msys2 I have installed Emacs, which I start with a custom cmd script with following environment variables set: {{{ SET PATH=C:\msys64\mingw64\bin;C:\msys64\usr\bin;%PATH% set XDG_DATA_DIRS=c:/msys64/mingw64/share set PKG_CONFIG_PATH=c:/msys64/mingw64/lib/pkgconfig set INCLUDE=c:/msys64/mingw64/include }}} The haskell I have is from Full Haskell Platform, https://haskell.org/platform/download/8.2.1/HaskellPlatform-8.2.1-full- x86_64-setup.exe Then I started eshell which I used to invoke this command: {{{ cabal install gi-gtk }}} Which after a while gave me following error. {{{ [91 of 95] Compiling GI.Pango.Objects.Layout ( GI\Pango\Objects\Layout.hs, dist\build\GI\Pango\Objects\Layout.o ) ghc.exe: panic! (the 'impossible' happened) (GHC version 8.2.1 for x86_64-unknown-mingw32): tcIfaceGlobal (local): not found You are in a maze of twisty little passages, all alike. While forcing the thunk for TyThing Layout which was lazily initialized by initIfaceCheck typecheckLoop, I tried to tie the knot, but I couldn't find Layout in the current type environment. If you are developing GHC, please read Note [Tying the knot] and Note [Type-checking inside the knot]. Consider rebuilding GHC with profiling for a better stack trace. Contents of current type environment: [] Call stack: CallStack (from HasCallStack): prettyCurrentCallStack, called at compiler\utils\Outputable.hs:1133:58 in ghc:Outputable callStackDoc, called at compiler\utils\Outputable.hs:1137:37 in ghc:Outputable pprPanic, called at compiler\iface\TcIface.hs:1696:23 in ghc:TcIface Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug cabal: Leaving directory 'C:\Users\Jacek\AppData\Local\Temp\cabal-tmp-8093 \gi-pango-1.0.15' Failed to install gi-gio-2.0.14 Build log ( C:\Users\Jacek\AppData\Roaming\cabal\logs\ghc-8.2.1\gi- gio-2.0.14-GKluzGq73QJBrHtRklhQDd.log ): Preprocessing library for gi-gio-2.0.14.. Building library for gi-gio-2.0.14.. }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14531> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 4

[GHC] #14543: Broken links all over the web
by GHC 30 Nov '17

30 Nov '17

#14543: Broken links all over the web -------------------------------------+------------------------------------- Reporter: SimonHengel | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.2.1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- Example: https://downloads.haskell.org/~ghc/latest/docs/html/users_guide /type-families.html -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14543> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 6