November 2014 - ghc-tickets

[GHC] #8457: -ffull-laziness does more harm than good
by GHC 08 Dec '17

08 Dec '17

#8457: -ffull-laziness does more harm than good ------------------------------------+------------------------------------- Reporter: errge | Owner: Type: bug | Status: new Priority: high | Milestone: 7.8.1 Component: Compiler | Version: 7.7 Keywords: | Operating System: Unknown/Multiple Architecture: Unknown/Multiple | Type of failure: None/Unknown Difficulty: Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | ------------------------------------+------------------------------------- In this bug report I'd like to argue that `-ffull-laziness` shouldn't be turned on automatically with either `-O` nor `-O2`, because it's dangerous and can cause serious memory leaks which are hard to debug or prevent. I'll also try to show that its optimization benefits are negligible. Actually, my benchmarks show that it's beneficial to turn it off even in the cases where we don't hit a space leak. We've met this issue last week, but it had been reported several times before: e.g. #917 and #5262. A typical example is the following: {{{ #!haskell main :: IO () main = task () >> task () task :: () -> IO () task () = printvalues [1..1000000 :: Int] printvalues :: [Int] -> IO () printvalues (x:xs) = print x >> printvalues xs printvalues [] = return () }}} We succeed with `-O0`, but fail with `-O`: {{{ errge@curry:~/tmp $ ~/tmp/ghc/inplace/bin/ghc-stage2 -v0 -O0 -fforce- recomp lazy && ./lazy +RTS -t >/dev/null <<ghc: 1620098744 bytes, 3117 GCs, 32265/42580 avg/max bytes residency (3 samples), 2M in use, 0.00 INIT (0.00 elapsed), 1.28 MUT (1.28 elapsed), 0.02 GC (0.02 elapsed) :ghc>> errge@curry:~/tmp $ ~/tmp/ghc/inplace/bin/ghc-stage2 -v0 -O -fforce-recomp lazy && ./lazy +RTS -t >/dev/null <<ghc: 1444098612 bytes, 2761 GCs, 3812497/13044272 avg/max bytes residency (7 samples), 28M in use, 0.00 INIT (0.00 elapsed), 1.02 MUT (1.03 elapsed), 0.12 GC (0.12 elapsed) :ghc>> }}} 28M? What the leak!? Well, it's `-ffull-laziness`: {{{ errge@curry:~/tmp $ ~/tmp/ghc/inplace/bin/ghc-stage2 -v0 -O -fno-full- laziness -fforce-recomp lazy && ./lazy +RTS -t >/dev/null <<ghc: 1484098612 bytes, 2835 GCs, 34812/42580 avg/max bytes residency (2 samples), 1M in use, 0.00 INIT (0.00 elapsed), 1.04 MUT (1.04 elapsed), 0.02 GC (0.02 elapsed) :ghc>> }}} We get constant space and the fastest run-time too, since we spare some cycles on GC. Note, that in this instance we are trying to explicity disable sharing by using `()` as a fake argument for the function. Also note, that this function may easily be a utility function in a larger code base or in a library, therefore it's impractical to say that you shouldn't use it twice "too close together". Quoting from the GHC user guide: {{{ -O2: Means: “Apply every non-dangerous optimisation, even if it means significantly longer compile times.” The avoided “dangerous” optimisations are those that can make runtime or space worse if you're unlucky. They are normally turned on or off individually. At the moment, -O2 is unlikely to produce better code than -O. }}} This seems to be false at the moment. We decided to make a broader investigation into this issue and wanted to know if we can disable this optimization without too much pain. Came up with this benchmark plan: - let's benchmark GHC, - compile all stages with -O, but hack the stage1 compiler to emit `-t` statistics for every file compiled, - gather these statistics while compiling the libraries and the stage2 compiler. On the second run we compile the stage1 compiler with `-O -fno-full-laziness`, but leave everything else unchanged in the environment. When we have both results of the compilation of ~1600 files, we match them up and compute the (logarithmic) ratio of CPU and memory difference between compilations, the final results for our benchmark. The results and the raw data can be found at https://github.com/errge/notlazy. The overall compilation time dropped from 26:20 to 25:12, which is a 4% improvement. Investigating the full matching shows that this overall result is from small improvements all around the place. The results plotted: - https://github.com/errge/notlazy/blob/master/cpu.png - https://github.com/errge/notlazy/blob/master/mem.png The graphs show the logarithmic (100*log_10(new/orig)) ratio of change in cpu and memory consumption. Therefore negative results mean that the new compilation method is faster. As can be seen on the CPU graph, in most of the cases the difference is negligible (actually smaller than what can be measured on small files, this is why we have the spike at 0). In overall we see a small improvement in CPU, and there are some outliers in both directions, but there are more drastic improvement cases than drastic regressions. On the memory graph the situation is much more close to zero. There is one big positive memory outlier: `DsListComp.lhs`. It uses 69M originally and now uses 103M. But compiles in 2 seconds both ways and there are files in the source tree which requires 400M to compile, so this is not an issue. After all this, I'd like to hear other opinions about just disabling this optimization in `-O` and `-O2` and leaving it as an option that can be turned on when needed, my reasons once more: - it's unsafe, - it's hard to debug when you hit its issues, - the optimization doesn't seem to be very productive, - it's always easy to force sharing, but it's not easy to force copying. Apparently a Haskell programmer should be lazy, but never fully lazy. Research done by Gergely Risko <errge> and Mihaly Barasz <klao>, confirmed on two different machines with no other running processes. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/8457> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 14

[GHC] #9118: Can't eta-reduce representational coercions
by GHC 19 Nov '17

19 Nov '17

#9118: Can't eta-reduce representational coercions ------------------------------------+------------------------------------- Reporter: goldfire | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.2 Keywords: | Operating System: Unknown/Multiple Architecture: Unknown/Multiple | Type of failure: None/Unknown Difficulty: Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | ------------------------------------+------------------------------------- When I say {{{ import Data.Type.Coercion import Data.Coerce import Data.Proxy eta2 :: Coercible (f a) (g a) => Proxy a -> Coercion f g eta2 _ = Coercion }}} I get {{{ Could not coerce from ‘f’ to ‘g’ because ‘f’ and ‘g’ are different types. arising from a use of ‘Coercion’ from the context (Coercible (f a) (g a)) bound by the type signature for eta2 :: Coercible (f a) (g a) => Proxy a -> Coercion f g at /Users/rae/temp/Roles.hs:5:9-56 In the expression: Coercion In an equation for ‘eta2’: eta2 _ = Coercion }}} Unlike the case for #9116, this one is ''not'' expressible in Core. (At least, I don't see a way to do it.) So, to do this, we would have to update Core and then update the constraint solver. I ''do'' think this would be type safe. But, I also think it's reasonable to wait for someone with a real use case to shout before doing this. The only use case I have is to be able to do this: {{{ -- says that a's parameter is representational class Rep (a :: k1 -> k2) where co :: Coercible x y => Coercible (a x) (a x) instance Rep f => Rep (Compose f) where ... }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9118> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 9

[GHC] #9052: Support a "stable heap" which doesn't get garbage collected
by GHC 13 Nov '17

13 Nov '17

#9052: Support a "stable heap" which doesn't get garbage collected ----------------------------------------------+---------------------------- Reporter: ezyang | Owner: Type: feature request | simonmar Priority: normal | Status: new Component: Runtime System | Milestone: Keywords: | Version: 7.9 Architecture: Unknown/Multiple | Operating System: Difficulty: Moderate (less than a day) | Unknown/Multiple Blocked By: | Type of failure: Related Tickets: | None/Unknown | Test Case: | Blocking: ----------------------------------------------+---------------------------- Simon Marlow was explaining the concept on IRC. The problem we're trying to solve is you might have some very long-lived pointer data-structures, and you don't want to waste time traversing this structure on a major GC when you know that it is still going to be alive. Here's the basic idea: 1. We have a new generation on the heap called the 'stable heap' (along- side the nursery, gen0, gen1, etc). By default, no objects are automatically promoted from the old gen to the stable heap. 2. A user can request an object be moved to the stable heap, creating a stable pointer which points to the object. 3. During garbage collection, all blocks in the stable heap are always considered reachable. We don't include the stable pointers in the root set (so we don't trace the stable heap), however, write barriers are processed as usual, and we promote any objects pointed to by the stable heap to the stable heap. 4. Users manually request the stable heap to be garbage-collected, usually after deallocating stable pointers. In short, the stable heap is just another generation, but one which (1) is only collected by user request, and (2) never receives objects from automatic promotion. This project fits in very nicely with the existing GC infrastructure, so if you understand how everything is laid out this should be reasonably easy to implement. Could be a good ticket for someone who wants to get their hands dirty with the GC. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9052> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 6

[GHC] #9420: Impredicative type instantiation without -XImpredicativeTypes
by GHC 11 Nov '17

11 Nov '17

#9420: Impredicative type instantiation without -XImpredicativeTypes -------------------------------------+------------------------------------- Reporter: goldfire | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.9 Keywords: | Operating System: Architecture: Unknown/Multiple | Unknown/Multiple Difficulty: Unknown | Type of failure: Blocked By: | None/Unknown Related Tickets: | Test Case: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Consider this module: {{{ {-# LANGUAGE RankNTypes #-} module Bug where rank2 :: ((forall a. a -> a) -> b) -> () rank2 _ = () foo :: () -> () foo x = x quux :: (forall a. a -> a) -> Int quux _ = 5 bar = foo . rank2 $ quux }}} The `(.)` in the definition of `bar` requires an impredicative instantiation -- that is, one of its type variables is instantiated to a forall-type. Yet, the module compiles without `-XImpredicativeTypes`. To confirm this behavior, the following is an excerpt from `-ddump-simpl`: {{{ Bug.bar = break<7>() (break<6>() GHC.Base.. @ () @ () @ ((forall a_a1Tj. a_a1Tj -> a_a1Tj) -> GHC.Types.Int) Bug.foo (Bug.rank2 @ GHC.Types.Int)) Bug.quux }}} Note the third type argument to `(.)`. Fixing this would be a breaking change that could affect users. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9420> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 4

[GHC] #9642: LANGUAGE pragma synonyms
by GHC 10 Nov '17

10 Nov '17

#9642: LANGUAGE pragma synonyms -------------------------------------+------------------------------------- Reporter: dreixel | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.9 Keywords: | Operating System: Architecture: Unknown/Multiple | Unknown/Multiple Difficulty: Unknown | Type of failure: Blocked By: | None/Unknown Related Tickets: | Test Case: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Here are the first 20 lines of a typical Haskell file of mine: {{{ {-# LANGUAGE EmptyDataDecls #-} {-# LANGUAGE FlexibleContexts #-} {-# LANGUAGE FlexibleInstances #-} {-# LANGUAGE LiberalTypeSynonyms #-} {-# LANGUAGE ScopedTypeVariables #-} {-# LANGUAGE RankNTypes #-} {-# LANGUAGE StandaloneDeriving #-} {-# LANGUAGE AutoDeriveTypeable #-} {-# LANGUAGE DeriveDataTypeable #-} {-# LANGUAGE DeriveGeneric #-} {-# LANGUAGE DefaultSignatures #-} {-# LANGUAGE OverlappingInstances #-} {-# LANGUAGE UndecidableInstances #-} {-# LANGUAGE MultiParamTypeClasses #-} {-# LANGUAGE TypeOperators #-} {-# LANGUAGE TypeFamilies #-} {-# LANGUAGE GADTs #-} {-# LANGUAGE DataKinds #-} {-# LANGUAGE PolyKinds #-} {-# LANGUAGE ConstraintKinds #-} }}} Over a couple of modules in a small package, I find myself copy-pasting these 20 lines, as either I will use these extensions, or they won't really cause any harm to the code I'm writing. As a way to simplify this, and reduce code duplication, I propose introducing LANGUAGE pragma synonyms: {{{ {-# LANGUAGE PedrosHaskell = EmptyDataDecls, FlexibleContexts, FlexibleInstances , LiberalTypeSynonyms, ScopedTypeVariables , RankNTypes, StandaloneDeriving, AutoDeriveTypeable , DeriveDataTypeable, DeriveGeneric, DefaultSignatures , OverlappingInstances, UndecidableInstances , MultiParamTypeClasses, TypeOperators, TypeFamilies , GADTs, DataKinds, PolyKinds, ConstraintKinds #-} }}} After writing this, in that same module or any module that imports it, I could just write: {{{ {-# LANGUAGE PedrosHaskell #-} }}} And all the pragmas above would be enabled. Besides making my life easier, I suspect this will also help increase modularity of certain packages. For example, packages could define a package-specific language pragma that should be enabled in order to use that package. Then, if the package maintainer upgrades one datatype to become a GADT, the pragma would be updated to include `GADTs`, and user code (that could now be requiring that pragma) would automatically enable `GADTs` too. Furthermore, it can also make the language standardisation process simpler. After all, {{{ {-# LANGUAGE Haskell2010 = PatternGuards, NoNPlusKPatterns, RelaxedPolyRec , EmptyDataDecls, ForeignFunctionInterface #-} }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9642> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 11

[GHC] #8238: Implement unloading of shared libraries
by GHC 09 Nov '17

09 Nov '17

#8238: Implement unloading of shared libraries ------------------------------------+------------------------------------- Reporter: simonmar | Owner: Type: task | Status: new Priority: normal | Milestone: 7.10.1 Component: Runtime System | Version: 7.7 Keywords: | Operating System: Unknown/Multiple Architecture: Unknown/Multiple | Type of failure: None/Unknown Difficulty: Unknown | Test Case: Blocked By: | Blocking: Related Tickets: 8039 | ------------------------------------+------------------------------------- In #8039 we added support for unloading static objects from the runtime linker, with the GC detecting when there are no references left. We could add this functionality for shared libraries too, using `dl_iterate_phdr`. @heisenbug's comment from #8039 with the relevant pointers: [http://eli.thegreenplace.net/2011/08/25/load-time-relocation-of-shared- libraries Eli Bendersky's article] suggests to use the [http://linux.die.net/man/3/dl_iterate_phdr dl_iterate_phdr] function for finding information about loaded libraries. Seems to be linux only. There is a [http://stackoverflow.com/questions/10009043/dl-iterate-phdr- equivalent-on-mac workaround on OSX], though, on !StackOverflow. And here is how Böhm-Demers-Weiser's GC implement a [https://github.com/ivmai/bdwgc/blob/master/dyn_load.c#L451 callback function for dl_iterate_phdr]. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/8238> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 6

[GHC] #8171: Extending ExtendedDefaultRules
by GHC 02 Nov '17

02 Nov '17

#8171: Extending ExtendedDefaultRules ------------------------------------+------------------------------------- Reporter: ekmett | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 Keywords: | Operating System: Unknown/Multiple Architecture: Unknown/Multiple | Type of failure: None/Unknown Difficulty: Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | ------------------------------------+------------------------------------- The current `ExtendedDefaultRules` is set up in a way that largely prevents it from being used for scenarios beyond what they get used for in GHCi today. It would be great if there was an extra class that could be included in the list of acceptable classes for Rule 3. {{{ class Defaulting a }}} The proposed change would be that Rule 3 under EDR can be be relaxed to: At least one of the classes Ci is numeric, or is `Show`, `Eq`, `Ord` or `Defaulting`. I went with `Defaulting` rather than `Default`, because `Default` is a common class in use across the ecosystem from the `data-default` package, and I wanted to avoid gratuitous fallout. This would permit projects like Shae Erisson's `ghclive` from last year that need to use defaulting to not have to shoe-horn unnecessary `Show` constraints onto functions. Even better would be for that class to be polykinded! That opens up possibilities for expert users to get defaulting to fire on arguments of other kinds, which is impossible under the current EDR. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/8171> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 14

[GHC] #9249: Link to "latest" user's guide
by GHC 01 Nov '17

01 Nov '17

#9249: Link to "latest" user's guide ------------------------------------+------------------------------------- Reporter: goldfire | Owner: Type: bug | Status: new Priority: low | Milestone: Component: Documentation | Version: 7.8.2 Keywords: | Operating System: Unknown/Multiple Architecture: Unknown/Multiple | Type of failure: None/Unknown Difficulty: Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | ------------------------------------+------------------------------------- When I search (say, on Google) for a GHC-related bit of documentation, I variously get referred to old manuals. There doesn't seem to a good way for us, the GHC devs, to avoid this. But, perhaps we can include some information in the header of the pages to allow visitors to link to the latest manual, or perhaps to several versions. This header should also include the version number of the GHC that the page is documenting; currently, this info is only in the URL. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9249> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 4

[GHC] #9112: data families with representational matching
by GHC 17 Oct '17

17 Oct '17

#9112: data families with representational matching ------------------------------------+------------------------------------- Reporter: jwlato | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.2 Keywords: | Operating System: Unknown/Multiple Architecture: Unknown/Multiple | Type of failure: None/Unknown Difficulty: Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | ------------------------------------+------------------------------------- Since ghc-7.8, the following is no longer possible: {{{ -- simplified example taken from the vector package class MVectorClass (v :: * -> * -> *) a where basicLength :: v s a -> Int data family MVector s a data instance MVector s Int -- implementation not important newtype Age = Age Int deriving (MVectorClass MVector) -- rejected }}} Following from discussion in #8177, to enable this ghc would need to support data families with representational matching, such that `MVector s Int` and `MVector s Age` are representationally equal. This has broken some code that previously worked, however as there are some workarounds I'm not sure how important it is. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9112> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 14

[GHC] #9274: GHC panic with UnliftedFFITypes+CApiFFI
by GHC 16 Oct '17

16 Oct '17

#9274: GHC panic with UnliftedFFITypes+CApiFFI -----------------------------------+--------------------------------------- Reporter: hvr | Owner: Type: bug | Status: new Priority: normal | Milestone: 7.10.1 Component: Compiler | Version: 7.6.3 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: Compile-time crash Unknown/Multiple | Test Case: Difficulty: Unknown | Blocking: Blocked By: | Related Tickets: | -----------------------------------+--------------------------------------- The following code fails to compile with at least GHC 7.6, 7.8 and HEAD: {{{#!hs import GHC.Prim import Foreign.C type GmpLimb = CULong type GmpSize = CLong -- mp_limb_t mpn_add_1 (mp_limb_t *rp, const mp_limb_t *s1p, mp_size_t n, mp_limb_t s2limb) foreign import capi unsafe "gmp.h mpn_add_1" c_mpn_add_1 :: MutableByteArray# s -> ByteArray# -> GmpSize -> GmpLimb -> IO GmpLimb }}} with the following compile error message: {{{ ghc: panic! (the 'impossible' happened) (GHC version 7.9.20140704 for x86_64-unknown-linux): toCType MutableByteArray# s Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug }}} Otoh, replacing `capi` with `ccall` makes the code compile just fine -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9274> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 7