March 2014 - ghc-tickets

[GHC] #8457: -ffull-laziness does more harm than good
by GHC 08 Dec '17

08 Dec '17

#8457: -ffull-laziness does more harm than good ------------------------------------+------------------------------------- Reporter: errge | Owner: Type: bug | Status: new Priority: high | Milestone: 7.8.1 Component: Compiler | Version: 7.7 Keywords: | Operating System: Unknown/Multiple Architecture: Unknown/Multiple | Type of failure: None/Unknown Difficulty: Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | ------------------------------------+------------------------------------- In this bug report I'd like to argue that `-ffull-laziness` shouldn't be turned on automatically with either `-O` nor `-O2`, because it's dangerous and can cause serious memory leaks which are hard to debug or prevent. I'll also try to show that its optimization benefits are negligible. Actually, my benchmarks show that it's beneficial to turn it off even in the cases where we don't hit a space leak. We've met this issue last week, but it had been reported several times before: e.g. #917 and #5262. A typical example is the following: {{{ #!haskell main :: IO () main = task () >> task () task :: () -> IO () task () = printvalues [1..1000000 :: Int] printvalues :: [Int] -> IO () printvalues (x:xs) = print x >> printvalues xs printvalues [] = return () }}} We succeed with `-O0`, but fail with `-O`: {{{ errge@curry:~/tmp $ ~/tmp/ghc/inplace/bin/ghc-stage2 -v0 -O0 -fforce- recomp lazy && ./lazy +RTS -t >/dev/null <<ghc: 1620098744 bytes, 3117 GCs, 32265/42580 avg/max bytes residency (3 samples), 2M in use, 0.00 INIT (0.00 elapsed), 1.28 MUT (1.28 elapsed), 0.02 GC (0.02 elapsed) :ghc>> errge@curry:~/tmp $ ~/tmp/ghc/inplace/bin/ghc-stage2 -v0 -O -fforce-recomp lazy && ./lazy +RTS -t >/dev/null <<ghc: 1444098612 bytes, 2761 GCs, 3812497/13044272 avg/max bytes residency (7 samples), 28M in use, 0.00 INIT (0.00 elapsed), 1.02 MUT (1.03 elapsed), 0.12 GC (0.12 elapsed) :ghc>> }}} 28M? What the leak!? Well, it's `-ffull-laziness`: {{{ errge@curry:~/tmp $ ~/tmp/ghc/inplace/bin/ghc-stage2 -v0 -O -fno-full- laziness -fforce-recomp lazy && ./lazy +RTS -t >/dev/null <<ghc: 1484098612 bytes, 2835 GCs, 34812/42580 avg/max bytes residency (2 samples), 1M in use, 0.00 INIT (0.00 elapsed), 1.04 MUT (1.04 elapsed), 0.02 GC (0.02 elapsed) :ghc>> }}} We get constant space and the fastest run-time too, since we spare some cycles on GC. Note, that in this instance we are trying to explicity disable sharing by using `()` as a fake argument for the function. Also note, that this function may easily be a utility function in a larger code base or in a library, therefore it's impractical to say that you shouldn't use it twice "too close together". Quoting from the GHC user guide: {{{ -O2: Means: “Apply every non-dangerous optimisation, even if it means significantly longer compile times.” The avoided “dangerous” optimisations are those that can make runtime or space worse if you're unlucky. They are normally turned on or off individually. At the moment, -O2 is unlikely to produce better code than -O. }}} This seems to be false at the moment. We decided to make a broader investigation into this issue and wanted to know if we can disable this optimization without too much pain. Came up with this benchmark plan: - let's benchmark GHC, - compile all stages with -O, but hack the stage1 compiler to emit `-t` statistics for every file compiled, - gather these statistics while compiling the libraries and the stage2 compiler. On the second run we compile the stage1 compiler with `-O -fno-full-laziness`, but leave everything else unchanged in the environment. When we have both results of the compilation of ~1600 files, we match them up and compute the (logarithmic) ratio of CPU and memory difference between compilations, the final results for our benchmark. The results and the raw data can be found at https://github.com/errge/notlazy. The overall compilation time dropped from 26:20 to 25:12, which is a 4% improvement. Investigating the full matching shows that this overall result is from small improvements all around the place. The results plotted: - https://github.com/errge/notlazy/blob/master/cpu.png - https://github.com/errge/notlazy/blob/master/mem.png The graphs show the logarithmic (100*log_10(new/orig)) ratio of change in cpu and memory consumption. Therefore negative results mean that the new compilation method is faster. As can be seen on the CPU graph, in most of the cases the difference is negligible (actually smaller than what can be measured on small files, this is why we have the spike at 0). In overall we see a small improvement in CPU, and there are some outliers in both directions, but there are more drastic improvement cases than drastic regressions. On the memory graph the situation is much more close to zero. There is one big positive memory outlier: `DsListComp.lhs`. It uses 69M originally and now uses 103M. But compiles in 2 seconds both ways and there are files in the source tree which requires 400M to compile, so this is not an issue. After all this, I'd like to hear other opinions about just disabling this optimization in `-O` and `-O2` and leaving it as an option that can be turned on when needed, my reasons once more: - it's unsafe, - it's hard to debug when you hit its issues, - the optimization doesn't seem to be very productive, - it's always easy to force sharing, but it's not easy to force copying. Apparently a Haskell programmer should be lazy, but never fully lazy. Research done by Gergely Risko <errge> and Mihaly Barasz <klao>, confirmed on two different machines with no other running processes. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/8457> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 14

[GHC] #8238: Implement unloading of shared libraries
by GHC 10 Nov '17

10 Nov '17

#8238: Implement unloading of shared libraries ------------------------------------+------------------------------------- Reporter: simonmar | Owner: Type: task | Status: new Priority: normal | Milestone: 7.10.1 Component: Runtime System | Version: 7.7 Keywords: | Operating System: Unknown/Multiple Architecture: Unknown/Multiple | Type of failure: None/Unknown Difficulty: Unknown | Test Case: Blocked By: | Blocking: Related Tickets: 8039 | ------------------------------------+------------------------------------- In #8039 we added support for unloading static objects from the runtime linker, with the GC detecting when there are no references left. We could add this functionality for shared libraries too, using `dl_iterate_phdr`. @heisenbug's comment from #8039 with the relevant pointers: [http://eli.thegreenplace.net/2011/08/25/load-time-relocation-of-shared- libraries Eli Bendersky's article] suggests to use the [http://linux.die.net/man/3/dl_iterate_phdr dl_iterate_phdr] function for finding information about loaded libraries. Seems to be linux only. There is a [http://stackoverflow.com/questions/10009043/dl-iterate-phdr- equivalent-on-mac workaround on OSX], though, on !StackOverflow. And here is how Böhm-Demers-Weiser's GC implement a [https://github.com/ivmai/bdwgc/blob/master/dyn_load.c#L451 callback function for dl_iterate_phdr]. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/8238> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 6

[GHC] #8171: Extending ExtendedDefaultRules
by GHC 02 Nov '17

02 Nov '17

#8171: Extending ExtendedDefaultRules ------------------------------------+------------------------------------- Reporter: ekmett | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 Keywords: | Operating System: Unknown/Multiple Architecture: Unknown/Multiple | Type of failure: None/Unknown Difficulty: Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | ------------------------------------+------------------------------------- The current `ExtendedDefaultRules` is set up in a way that largely prevents it from being used for scenarios beyond what they get used for in GHCi today. It would be great if there was an extra class that could be included in the list of acceptable classes for Rule 3. {{{ class Defaulting a }}} The proposed change would be that Rule 3 under EDR can be be relaxed to: At least one of the classes Ci is numeric, or is `Show`, `Eq`, `Ord` or `Defaulting`. I went with `Defaulting` rather than `Default`, because `Default` is a common class in use across the ecosystem from the `data-default` package, and I wanted to avoid gratuitous fallout. This would permit projects like Shae Erisson's `ghclive` from last year that need to use defaulting to not have to shoe-horn unnecessary `Show` constraints onto functions. Even better would be for that class to be polykinded! That opens up possibilities for expert users to get defaulting to fire on arguments of other kinds, which is impossible under the current EDR. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/8171> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 14

[GHC] #8919: Why is xhtml library installed but not exported to users?
by GHC 20 Sep '17

20 Sep '17

#8919: Why is xhtml library installed but not exported to users? ------------------------------------+------------------------------------- Reporter: simons | Owner: Type: bug | Status: new Priority: normal | Milestone: 7.8.1 Component: Build System | Version: 7.6.3 Keywords: | Operating System: Unknown/Multiple Architecture: Unknown/Multiple | Type of failure: Other Difficulty: Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | ------------------------------------+------------------------------------- GHC 7.8.1-rc2 installs the xhtml library, but doesn't export it, i.e. it's completely invisible to users. Is there a particular reason why this setup is necessary? If GHC needs that library internally anywhere, would it be possible to link it statically and *not* install it? Or, if that's not possible, if the library needs to be installed, please make it visible to users. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/8919> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 27

[GHC] #8091: retry# lacks strictness information
by GHC 13 Sep '17

13 Sep '17

#8091: retry# lacks strictness information ------------------------------------+------------------------------------- Reporter: parcs | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 Keywords: | Operating System: Unknown/Multiple Architecture: Unknown/Multiple | Type of failure: None/Unknown Difficulty: Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | ------------------------------------+------------------------------------- The `retry#` primop should be marked as returning bottom since it never actually returns anything as far as the simplifier is concerned. This change would help the simplifier eliminate unreachable code inside STM transactions. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/8091> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 13

[GHC] #8532: Hyperbolic arc cosine fails on (-1) :: Complex r.
by GHC 13 Sep '17

13 Sep '17

#8532: Hyperbolic arc cosine fails on (-1) :: Complex r. -------------------------------------+------------------------------------- Reporter: leftaroundabout | Owner: Type: bug | Status: new Priority: low | Milestone: Component: libraries/base | Version: 7.6.3 Keywords: | Operating System: Unknown/Multiple Architecture: Unknown/Multiple | Type of failure: Incorrect result Difficulty: Easy (less than 1 | at runtime hour) | Test Case: acosh(-1) :: Blocked By: | Complex Double Related Tickets: | Blocking: -------------------------------------+------------------------------------- When allowing for complex results, the hyperbolic arc cosine is continuously defined on all ℝ. In the {{{(x < (-1))}}} real ray of the complex plane, {{{acosh}}} equals {{{\z -> i * pi + acosh(abs z)}}}, which works fine for almost all arguments. Thus, {{{acosh (-1)}}} should equal {{{i * pi}}}; however due to the implementation as {{{ acosh z = log (z + (z+1) * sqrt ((z-1)/(z+1))) }}} where the denominator in the root becomes zero at {{{z = -1}}}, this comes out as {{{NaN :+ NaN}}}. Could be fixed trivially by adding a special case {{{ acosh ((-1):+0) = 0:+pi }}} to the {{{instance (RealFloat a) => Floating (Complex a)}}} in {{{Data.Complex}}}. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/8532> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 13

[GHC] #8583: Associated pattern synonyms
by GHC 11 Sep '17

11 Sep '17

#8583: Associated pattern synonyms ------------------------------------+------------------------------------- Reporter: cactus | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: Keywords: | Operating System: Unknown/Multiple Architecture: Unknown/Multiple | Type of failure: None/Unknown Difficulty: Unknown | Test Case: Blocked By: 5144 | Blocking: Related Tickets: | ------------------------------------+------------------------------------- The PatternSynonyms wiki page has a section on (typeclass-)associated pattern synonyms: {{{ class ListLike l where pattern Nil :: l a pattern Cons :: a -> l a -> a isNil :: l a -> Bool isNil Nil = True isNil (Cons _ _) = False append :: l a -> l a -> l a instance ListLike [] where pattern Nil = [] pattern Cons x xs = x:xs append = (++) headOf :: (ListLike l) => l a -> Maybe a headOf Nil = Nothing headOf (Cons x _) = Just x }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/8583> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 18

[GHC] #8317: Optimize tagToEnum# at Core level
by GHC 29 Aug '17

29 Aug '17

#8317: Optimize tagToEnum# at Core level ------------------------------------+------------------------------------- Reporter: jstolarek | Owner: Type: task | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.7 Keywords: | Operating System: Unknown/Multiple Architecture: Unknown/Multiple | Type of failure: None/Unknown Difficulty: Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #6135 | ------------------------------------+------------------------------------- Old comparison primops that returned Bool were implemented by inserting call to `tagToEnum#` in the code generator. This call to `tagToEnum#` was later optimized away by a special case in the code geenrator (see [[GhcFile(compiler/codeGen/StgCmmExpr.hs)]], Note [case on bool])). Now that we have new comparison primops (see #6135) we no longer insert `tagToEnum#` in the code generator - all uses of `tagToEnum#` come from explicit calls in a source program. But we still have that special case in the code generator to optimize away `tagToEnum#`. We should drop that special case in the code generator and handle scrutinizing of `tagToEnum#` at the Core level by: 1. removing call to `tagToEnum#` in the scrutinee 2. adding calls to `dataToTag#` in case branches 3. constant-folding inserted `dataToTag#` Here is an example. This Haskell code: {{{ if tagToEnum# (a ># b) then E1 else E2 }}} will be translated to this Core: {{{ case tagToEnum# (a ># b) of True -> E1 False -> E2 }}} and optimized like this: {{{ case a ># b of dataToTag# True -> E1 dataToTag# False -> E2 }}} ====> {{{ case a ># b of 1 -> E1 0 -> E2 }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/8317> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 8

[GHC] #8472: Primitive string literals prevent optimization
by GHC 16 Aug '17

16 Aug '17

#8472: Primitive string literals prevent optimization ------------------------------+-------------------------------------------- Reporter: akio | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 Keywords: | Operating System: Linux Architecture: x86_64 | Type of failure: Runtime performance bug (amd64) | Test Case: Difficulty: Unknown | Blocking: Blocked By: | Related Tickets: | ------------------------------+-------------------------------------------- Using an Addr# literal seems to result in less aggressive optimization. If I compile the attached program like this: {{{ ghc -O2 -fforce-recomp -ddump-simpl addr.hs }}} the code is optimized nicely. Everything are inlined into {{{t}}}, intermediate pairs are eliminated, etc. However, when I replace the Int# literals in the code with Addr# literals by defining the {{{ADDR}}} macro: {{{ ghc -O2 -fforce-recomp -ddump-simpl -DADDR addr.hs }}} GHC now creates 2 extra top-level bindings, each of which allocates a pair. I don't see why the presence of Addr# literals should prevent inlining, so I'm reporting a bug. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/8472> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 24

[GHC] #8147: Exponential behavior in instance resolution on fixpoint-of-sum
by GHC 07 Aug '17

07 Aug '17

#8147: Exponential behavior in instance resolution on fixpoint-of-sum -------------------------+------------------------------------------------- Reporter: | Owner: jkoppel | Status: new Type: bug | Milestone: Priority: | Version: 7.6.3 normal | Operating System: Unknown/Multiple Component: | Type of failure: Compile-time performance bug Compiler | Test Case: Keywords: | Blocking: performance, | Architecture: | Unknown/Multiple | Difficulty: | Unknown | Blocked By: | Related Tickets: | -------------------------+------------------------------------------------- Doing instance resolution on a fixpoint-of-sum type takes a very long time. This is possibly the same issue as issue #5642. These are the numbers I see for various n: {{{ 10 : 0.329s 20 : 0.479s 40 : 0.935s 80 : 2.821s 160 : 11.694s 320 : 1m30.39s 640: Ran for over 1 hour without terminating }}} This uses a couple of attached support files. Apologies for not being able to reduce further. {{{ -- Test.hs {-# LANGUAGE MultiParamTypeClasses, FlexibleInstances, UndecidableInstances, OverlappingInstances, TypeOperators, DeriveFunctor, TemplateHaskell #-} module Test where import Control.Monad import Lib import TH {- With n=3, produces data X1 e = X1 e deriving (Functor) data X2 e = X2 e deriving (Functor) data X3 e = X3 e deriving (Functor) type X = X1 :+: X2 :+: X3 -} $(let n = 320 in liftM concat $ sequence [liftM concat $ mapM mkDec $ map (('X':).show) [1..n] , makeSumType "X" (map (('X':).show) [1..n])]) data Y0 e = Y0 e deriving ( Functor ) type X' = Y0 :+: X class Lift f g where lift' :: f (Fix g) -> Fix g instance (Lift f g, Lift f' g) => Lift (f :+: f') g where lift' x = case x of L e -> lift' e R e -> lift' e instance (Functor f, f :<: g) => Lift f g where lift' = In . inj cata :: (Functor f) => (f a -> a) -> Fix f -> a cata f = f . fmap (cata f) . out lift :: Fix X -> Fix X' lift = cata lift' }}} Virtually all the time is spent in compiling lift. For example, with n=640, commenting out lift makes it compile in around 2 seconds. Interestingly, when I add the following code, compilation times only increase by 10-20%. In the original code where I encountered this issue, doing so doubles compilation time. {{{ instance Lift Y0 X where lift' = undefined lower :: Fix X' -> Fix X lower = cata lift' }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/8147> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 6