March 2018 - ghc-tickets

[GHC] #14854: The size of FastString table is suboptimal for large codebases
by GHC 28 Oct '18

28 Oct '18

#14854: The size of FastString table is suboptimal for large codebases -------------------------------------+------------------------------------- Reporter: niteria | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: Compile-time Unknown/Multiple | performance bug Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- Context: I was profiling a no-op rebuild (build once, do incremental build with nothing changed) of a large code base. It takes `35.829s`, and according to the profile about `20%` of the time is spent in `mkFastStringByteString`. I run it with `-dfaststring-stats +RTS -s` and got: {{{ FastString stats: size: 4091 entries: 829959 longest chain: 268 has z-encoding: 0% 50,011,906,640 bytes allocated in the heap 7,936,277,664 bytes copied during GC 3,106,526,336 bytes maximum residency (8 sample(s)) 123,108,224 bytes maximum slop 10169 MB total memory in use (0 MB lost due to fragmentation) Tot time (elapsed) Avg pause Max pause Gen 0 117 colls, 37 par 32.390s 2.293s 0.0196s 0.0911s Gen 1 8 colls, 3 par 19.722s 2.266s 0.2832s 0.6936s Parallel GC work balance: 60.62% (serial 0%, perfect 100%) TASKS: 93 (1 bound, 92 peak workers (92 total), using -N30) SPARKS: 0 (0 converted, 0 overflowed, 0 dud, 0 GC'd, 0 fizzled) INIT time 0.009s ( 0.009s elapsed) MUT time 277.370s ( 31.261s elapsed) GC time 52.113s ( 4.558s elapsed) EXIT time 0.005s ( 0.001s elapsed) Total time 329.497s ( 35.829s elapsed) Alloc rate 180,307,295 bytes per MUT second Productivity 84.2% of total user, 87.3% of total elapsed }}} That's pretty bad, fitting `800k` elements in `4k` buckets is hard. To confirm this indeed could be improved I changed the size to `2^20` (`1048576`) and also removed division in `hashStr` function (it come up on `perf` profile), like this: {{{ hashStr :: Ptr Word8 -> Int -> Int -- use the Addr to produce a hash value between 0 & m (inclusive) hashStr (Ptr a#) (I# len#) = loop 0# 0# where loop h n | isTrue# (n ==# len#) = I# h | otherwise = loop h2 (n +# 1#) where !c = ord# (indexCharOffAddr# a# n) -- !h2 = (c +# (h *# 128#)) `remInt#` -- hASH_TBL_SIZE# -- NOTE: below is multiplication by 127 (a prime) and division by 2^20, it's by no means the best hashing function !h2 = (c +# ((h `uncheckedIShiftL#` 7#) -# h)) `andI#` hASH_TBL_SIZE_MASK# }}} Here's what I ended up with: {{{ FastString stats: size: 1048576 entries: 829959 longest chain: 8 has z-encoding: 0% 51,012,218,048 bytes allocated in the heap 8,948,377,768 bytes copied during GC 3,543,797,248 bytes maximum residency (8 sample(s)) 139,428,352 bytes maximum slop 10502 MB total memory in use (0 MB lost due to fragmentation) Tot time (elapsed) Avg pause Max pause Gen 0 115 colls, 39 par 32.261s 2.677s 0.0233s 0.2225s Gen 1 8 colls, 3 par 21.786s 2.329s 0.2911s 0.5737s Parallel GC work balance: 48.75% (serial 0%, perfect 100%) TASKS: 93 (1 bound, 92 peak workers (92 total), using -N30) SPARKS: 0 (0 converted, 0 overflowed, 0 dud, 0 GC'd, 0 fizzled) INIT time 0.008s ( 0.008s elapsed) MUT time 65.508s ( 23.791s elapsed) GC time 54.047s ( 5.006s elapsed) EXIT time 0.007s ( 0.001s elapsed) Total time 119.570s ( 28.806s elapsed) Alloc rate 778,712,134 bytes per MUT second Productivity 54.8% of total user, 82.6% of total elapsed }}} The realtime improvement is nice (`-20%`), but even nicer is the "serial" time improvement, from `329.497s`to `119.570s` (`2.75x` speedup). This is on modified 8.0.2, but the related code looks about the same on HEAD. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14854> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 11

[GHC] #14465: Performance of Natural
by GHC 28 Oct '18

28 Oct '18

#14465: Performance of Natural -------------------------------------+------------------------------------- Reporter: Bodigrim | Owner: (none) Type: feature | Status: new request | Priority: normal | Milestone: Component: Compiler | Version: 8.2.1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- Recently I tried to use `Natural` instead of `Integer` in one of my projects. I expected no difference or even a minor performance boost (since `Natural` does not have to worry about a sign). But in fact it caused a slowdown. A constant of type `Integer` will be evaluated to a low-level representation (`S#` / `Jp#` / `Jn#`) during `CorePrep` stage. Nothing of this kind happens to constant values of type `Natural`: {{{#!hs import Numeric.Natural one :: Natural one = fromInteger 1 }}} is translated to {{{#!hs one1 :: Integer one1 = 1 -- will be converted by CorePrep to S# 1# one :: Natural one = case one1 of { S# i#_a2c6 -> case tagToEnum# (>=# i#_a2c6 0#) of { False -> underflowError; True -> NatS# (int2Word# i#_a2c6) }; Jp# dt_a2cg -> case uncheckedIShiftRL# (sizeofByteArray# dt_a2cg) 3# of { __DEFAULT -> case sizeofByteArray# dt_a2cg of { __DEFAULT -> NatJ# dt_a2cg; 0# -> underflowError }; 1# -> case indexWordArray# dt_a2cg 0# of wild2_a2ck { __DEFAULT -> NatS# wild2_a2ck } }; Jn# ipv_a2cn -> underflowError } }}} This is not bad itself, if `one` is a top-level definition. At the end of the day a thunk will be replaced by its value, computed exactly once. But suppose we have written {{{#!hs import Numeric.Natural plusOne :: Natural -> Natural plusOne n = n + 1 }}} The corresponding Core looks this way: {{{#!hs plusOne :: Natural -> Natural plusOne = \ (n_auS :: Natural) -> case 1 of { S# i#_a2dA -> case tagToEnum# (>=# i#_a2dA 0#) of { False -> case underflowError of wild2_00 { }; True -> plusNatural n_auS (NatS# (int2Word# i#_a2dA)) }; Jp# dt_a2dI -> case uncheckedIShiftRL# (sizeofByteArray# dt_a2dI) 3# of { __DEFAULT -> case sizeofByteArray# dt_a2dI of { __DEFAULT -> plusNatural n_auS (NatJ# dt_a2dI); 0# -> case underflowError of wild4_00 { } }; 1# -> case indexWordArray# dt_a2dI 0# of wild2_a2dM { __DEFAULT -> plusNatural n_auS (NatS# wild2_a2dM) } }; Jn# ipv_a2dP -> case underflowError of wild1_00 { } } }}} It looks expensive to pattern match `1` repeatedly, at every call to `plusOne`. Another deficiency of `Natural` is that no constant folding is done. Even `2 * 2` results in 50 lines of Core: {{{#!hs twoTimesTwo2 :: Integer twoTimesTwo2 = 2 twoTimesTwo1 :: Natural twoTimesTwo1 = case twoTimesTwo2 of { S# i#_a2u3 -> case tagToEnum# (>=# i#_a2u3 0#) of { False -> underflowError; True -> NatS# (int2Word# i#_a2u3) }; Jp# dt_a2ub -> case uncheckedIShiftRL# (sizeofByteArray# dt_a2ub) 3# of { __DEFAULT -> case sizeofByteArray# dt_a2ub of { __DEFAULT -> NatJ# dt_a2ub; 0# -> underflowError }; 1# -> case indexWordArray# dt_a2ub 0# of wild2_a2uf { __DEFAULT -> NatS# wild2_a2uf } }; Jn# ipv_a2ui -> underflowError } twoTimesTwo :: Natural twoTimesTwo = case twoTimesTwo2 of { S# i#_a2u3 -> case tagToEnum# (>=# i#_a2u3 0#) of { False -> case underflowError of wild2_00 { }; True -> $fNumNatural_$c* twoTimesTwo1 (NatS# (int2Word# i#_a2u3)) }; Jp# dt_a2ub -> case uncheckedIShiftRL# (sizeofByteArray# dt_a2ub) 3# of { __DEFAULT -> case sizeofByteArray# dt_a2ub of { __DEFAULT -> $fNumNatural_$c* twoTimesTwo1 (NatJ# dt_a2ub); 0# -> case underflowError of wild4_00 { } }; 1# -> case indexWordArray# dt_a2ub 0# of wild2_a2uf { __DEFAULT -> $fNumNatural_$c* twoTimesTwo1 (NatS# wild2_a2uf) } }; Jn# ipv_a2ui -> case underflowError of wild1_00 { } } }}} This is not surprising, since constant folding of `Integer` works only due to a special Core literal `LitInteger` and a set of hardcoded `PrelRules.builtinIntegerRules` (and `NOINLINE` pragmas in `GHC.Integer.Type`). ---- Is it reasonable to make `Natural` a first-class Core citizen? I suppose a new `LitNatural` node and decent amount of copy-paste will do. Or, possibly, we may reuse `LitInteger Integer Type` with appropriate `Type` to avoid some duplication of code. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14465> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 4

[GHC] #9163: Ptr should have a phantom role
by GHC 28 Oct '18

28 Oct '18

#9163: Ptr should have a phantom role ------------------------------------+------------------------------------- Reporter: simonpj | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.2 Keywords: | Operating System: Unknown/Multiple Architecture: Unknown/Multiple | Type of failure: None/Unknown Difficulty: Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | ------------------------------------+------------------------------------- In `GHC.Ptr` we see {{{ type role Ptr representational data Ptr a = Ptr Addr# deriving (Eq, Ord) }}} with no comments. Why is `Ptr` representational? In the same module we have `castPtr`: {{{ castPtr :: Ptr a -> Ptr b castPtr (Ptr addr) = Ptr addr }}} which unpacks and repacks a `Ptr`. If `Ptr` was phantom, we could use `coerce`. And that in turn would actually make a lot of code more efficient – there are lots of calls to `castPtr`. Specifically, in `nofib`, I tried implementing `castPtr` with `unsafeCoerce`. Then I found: * 12% less allocation in `reverse-complem` * 7.3% less allocation in `fasta`. * Binary sizes fell 0.1%. Both these benchmarks are ones that do a ''lot'' of I/O, and it turns out that `GHC.IO.Handle.Text.$wa1` has a `castPtr` that (all by itself) accounts for 12% of `reverse-complem`'s total allocation! So making `castPtr` free is good. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9163> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 41

[GHC] #13809: TH-reified data family instances have a paucity of kinds
by GHC 27 Oct '18

27 Oct '18

#13809: TH-reified data family instances have a paucity of kinds -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Template | Version: 8.0.1 Haskell | Keywords: TypeFamilies | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- Consider this data family (and instances): {{{#!hs {-# LANGUAGE TypeFamilies #-} module Foo where data family Foo a data instance Foo ((f :: * -> *) (a :: *)) data instance Foo ((f :: (* -> *) -> *) (a :: (* -> *))) }}} These are two data family instances that GHC distinguishes by the kinds of their type parameters. However, Template Haskell does not give me the same insight that GHC has, because if I call `Language.Haskell.TH.reify ''Foo`, I get this: {{{#!hs FamilyI (DataFamilyD Foo.Foo [ KindedTV a_6989586621679025989 StarT ] (Just StarT)) [ DataInstD [] Foo.Foo [ AppT (VarT f_6989586621679026001) (VarT a_6989586621679026000) ] Nothing [] [] , DataInstD [] Foo.Foo [ AppT (VarT f_6989586621679026007) (VarT a_6989586621679026006) ] Nothing [] [] ] }}} Note that neither `f` nor `a` have a kind signature in either instance! This makes it completely impossible to tell which is which (aside from the order, which is brittle). It would make my life a lot easier if TH were to include kind signatures for each type variable in a data family instance. I can see two ways to accomplish this: 1. Include a `[TyVarBndr]` field in `DataInstD` and `NewtypeInstD` where each `TyVarBndr` is a `KindedTV`. 2. Walk over the `Type`s in a `DataInstD`/`NewtypeInstD` and ensure that every occurrence of a `VarT` is surrounded with `SigT` to indicate its kind. While (1) is arguably the cleaner solution, since it makes the kinds easy to discover, it is a breaking change. Therefore, I'm inclined to implement option (2) instead. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13809> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 9

[GHC] #14268: Implement Explicit Foralls Proposal
by GHC 27 Oct '18

27 Oct '18

#14268: Implement Explicit Foralls Proposal -------------------------------------+------------------------------------- Reporter: johnleo | Owner: (none) Type: task | Status: new Priority: normal | Milestone: 8.4.1 Component: Compiler | Version: 8.2.1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- Implement Richard Eisenberg's Explicit Foralls proposal. https://github.com/ghc-proposals/ghc-proposals/pull/55 For details see the proposal: https://github.com/goldfirere/ghc-proposals/blob/instance- forall/proposals/0000-instance-foralls.rst -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14268> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 13

[GHC] #11238: Redesign the dynamic linking on ELF systems
by GHC 26 Oct '18

26 Oct '18

#11238: Redesign the dynamic linking on ELF systems -------------------------------------+------------------------------------- Reporter: trommler | Owner: trommler Type: task | Status: new Priority: normal | Milestone: 8.2.1 Component: Runtime | Version: 7.10.3 System (Linker) | Keywords: | Operating System: Linux Architecture: | Type of failure: Other Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- This ticket is to follow up on my comment on #11042. Let's give dynamic linking on ELF systems a second chance. I propose the following steps: 1. Collect required features 1. Write up and discuss and agree on a specification 1. Create regression tests for all features 1. Design and implement a new dynamic linker for GHCi 1. (optional) Make it work on Mac OS X too -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/11238> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 10

[GHC] #10353: Haddock for Data.List should list instances
by GHC 26 Oct '18

26 Oct '18

#10353: Haddock for Data.List should list instances -------------------------------------+------------------------------------- Reporter: goldfire | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: | Version: 7.10.1 Documentation | Operating System: Unknown/Multiple Keywords: | Type of failure: None/Unknown Architecture: | Blocked By: Unknown/Multiple | Related Tickets: Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- I'd like to know the set of classes for which there are instances for `[]`. So, I naturally looked in the documentation for `Data.List`. However, because `Data.List` doesn't export `[]`, this list isn't there. Can this be fixed? (It's an honest question -- because of `[]`'s special syntax, I could see this fix being non-trivial.) -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10353> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 1

[GHC] #13913: Can't apply higher-ranked kind in type family
by GHC 26 Oct '18

26 Oct '18

#13913: Can't apply higher-ranked kind in type family -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.1 (Type checker) | Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: GHC rejects Unknown/Multiple | valid program Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- This code doesn't typecheck due to `F2`: {{{#!hs {-# LANGUAGE RankNTypes #-} {-# LANGUAGE TypeFamilies #-} {-# LANGUAGE TypeInType #-} module Bug where import Data.Kind f :: (forall (a :: Type). a -> a) -> Bool f g = g True type F1 (g :: forall (a :: Type). a -> a) = g True type family F2 (g :: forall (a :: Type). a -> a) :: Bool where F2 g = g True }}} {{{ $ /opt/ghc/8.2.1/bin/ghci Bug.hs GHCi, version 8.2.0.20170623: http://www.haskell.org/ghc/ :? for help Loaded GHCi configuration from /home/rgscott/.ghci [1 of 1] Compiling Bug ( Bug.hs, interpreted ) Bug.hs:14:6: error: • Expected kind ‘forall a. a -> a’, but ‘g’ has kind ‘Bool -> Bool’ • In the first argument of ‘F2’, namely ‘g’ In the type family declaration for ‘F2’ | 14 | F2 g = g True | ^ }}} This is surprising to me, since `F2` seems like the type family counterpart to `f` and `F1`, both of which typecheck. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13913> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 4

[GHC] #12160: MonadFail instance for (Either String)?
by GHC 25 Oct '18

25 Oct '18

#12160: MonadFail instance for (Either String)? -------------------------------------+------------------------------------- Reporter: lexi.lambda | Owner: Type: feature | Status: new request | Priority: normal | Milestone: Component: libraries | Version: 8.0.1 (other) | Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- I think one of the cool things about breaking out `MonadFail` would be the opportunity to introduce an instance for `Either String`, which isn’t possible to do with plain `Monad` without overlapping instances. The instance itself would be completely trivial: {{{#!hs instance MonadFail (Either String) where fail = Left }}} The only possible reason to ''not'' do this, as far as I can tell, would be because it requires `FlexibleInstances`. This already seems to be used for a few instances in `base`, and it seems like it would be extremely useful, so I would really appreciate if such an instance was defined. I’d be happy to submit a patch to add it, but I wasn’t sure if this would be an uncontroversial change or not. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/12160> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 15

[GHC] #14332: Deriving clauses can have forall types
by GHC 25 Oct '18

25 Oct '18

#14332: Deriving clauses can have forall types -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.2.1 (Type checker) | Keywords: deriving | Operating System: Unknown/Multiple Architecture: | Type of failure: GHC accepts Unknown/Multiple | invalid program Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- I made a horrifying discovery today: GHC accepts this code! {{{#!hs {-# LANGUAGE DeriveAnyClass #-} {-# LANGUAGE MultiParamTypeClasses #-} {-# LANGUAGE RankNTypes #-} {-# OPTIONS_GHC -ddump-deriv #-} module Bug1 where class C a b data D a = D deriving ((forall a. C a)) }}} {{{ GHCi, version 8.2.1: http://www.haskell.org/ghc/ :? for help Loaded GHCi configuration from /home/rgscott/.ghci [1 of 1] Compiling Bug1 ( Bug.hs, interpreted ) ==================== Derived instances ==================== Derived class instances: instance Bug1.C a1 (Bug1.D a2) where Derived type family instances: Ok, 1 module loaded. }}} It gets even worse with this example: {{{#!hs {-# LANGUAGE DeriveGeneric #-} {-# LANGUAGE RankNTypes #-} {-# LANGUAGE TypeInType #-} {-# OPTIONS_GHC -ddump-deriv -fprint-explicit-kinds #-} module Bug1 where import Data.Kind import GHC.Generics data Proxy (a :: k) = Proxy deriving ((forall k. (Generic1 :: (k -> Type) -> Constraint))) }}} {{{ GHCi, version 8.2.1: http://www.haskell.org/ghc/ :? for help Loaded GHCi configuration from /home/rgscott/.ghci [1 of 1] Compiling Bug1 ( Bug.hs, interpreted ) ==================== Derived instances ==================== Derived class instances: instance GHC.Generics.Generic1 k (Bug1.Proxy k) where GHC.Generics.from1 x_a3ip = GHC.Generics.M1 (case x_a3ip of { Bug1.Proxy -> GHC.Generics.M1 GHC.Generics.U1 }) GHC.Generics.to1 (GHC.Generics.M1 x_a3iq) = case x_a3iq of { (GHC.Generics.M1 GHC.Generics.U1) -> Bug1.Proxy } Derived type family instances: type GHC.Generics.Rep1 k_a2mY (Bug1.Proxy k_a2mY) = GHC.Generics.D1 k_a2mY ('GHC.Generics.MetaData "Proxy" "Bug1" "main" 'GHC.Types.False) (GHC.Generics.C1 k_a2mY ('GHC.Generics.MetaCons "Proxy" 'GHC.Generics.PrefixI 'GHC.Types.False) (GHC.Generics.U1 k_a2mY)) Ok, 1 module loaded. }}} In this example, the `forall`'d `k` from the `deriving` clause is discarded and then unified with the `k` from `data Proxy (a :: k)`. All of this is incredibly unsettling. We really shouldn't be allowing `forall` types in `deriving` clauses in the first place. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14332> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 18