September 2017 - ghc-tickets

[GHC] #10535: Float out causes major space leak
by GHC 08 Dec '17

08 Dec '17

#10535: Float out causes major space leak -------------------------------------+------------------------------------- Reporter: AlexET | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Blocked By: Test Case: | Related Tickets: Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- This issue originally was on haskell-cafe https://mail.haskell.org/pipermail/haskell-cafe/2015-June/120113.html The following code has a space leak {{{ #!hs numbers = [1..200] replicateM' :: Monad m => Int -> m a -> m [a] replicateM' 0 xs = return [] replicateM' n xs = do a <- xs b <- replicateM' (n-1) xs return (a:b) test :: [[Int]] test = replicateM' 4 numbers main = print test }}} The recursive call in replicateM' gets floated out. This then causes a very large (200^3 elements) list to be kept in memory instead of being lazily produced and consumed each time. A similar thing happens to the replicateM in Control.Monad but it is harder to spot as a lot of specialisation and fusion goes on before the float-out happens This is similar to #7367 but in this case we have a huge space-leak instead of just adding allocations. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10535> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 7

[GHC] #14254: The Binary instance for TypeRep smells a bit expensive
by GHC 01 Dec '17

01 Dec '17

#14254: The Binary instance for TypeRep smells a bit expensive -------------------------------------+------------------------------------- Reporter: dfeuer | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.2.1 Keywords: Typeable | Operating System: Unknown/Multiple Architecture: | Type of failure: Compile-time Unknown/Multiple | performance bug Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- In particular, `get` uses `getSomeTypeRep`. `getSomeTypeRep`, in turn, calls `typeRepKind` through its recursion. But `typeRepKind` is itself recursive, fully inspecting the spine of its argument. That smells quadratic to me. The solution, I believe, is to change the type of `getSomeTypeRep` to `BinHandle -> IO (SomeTypeRep, TypeRep Type)`. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14254> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 19

[GHC] #13941: STG linter's type equality can loop
by GHC 28 Nov '17

28 Nov '17

#13941: STG linter's type equality can loop -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.2.2 Component: Compiler | Version: 8.0.1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- `stgEqType` will currently loop when given the following, {{{#!hs stgEqType (a -> IO ()) (a -> IO ()) }}} The problem is that the `(->)` tycon now takes runtime rep arguments. To see why, let's look at the (paraphrased) implementation, {{{#!hs stgEqType orig_ty1 orig_ty2 = gos (typePrimRep orig_ty1) (typePrimRep orig_ty2) where gos :: [PrimRep] -> [PrimRep] -> Bool gos [_] [_] = go orig_ty1 orig_ty2 gos reps1 reps2 = reps1 == reps2 go :: UnaryType -> UnaryType -> Bool go ty1 ty2 | Just (tc1, tc_args1) <- splitTyConApp_maybe ty1 , Just (tc2, tc_args2) <- splitTyConApp_maybe ty2 , let res = if tc1 == tc2 then equalLength tc_args1 tc_args2 && and (zipWith (gos `on` typePrimRep) tc_args1 tc_args2) else (isFamilyTyCon tc1 || isFamilyTyCon tc2) = res | otherwise = True -- Conservatively say "fine". }}} Our example will begin by looking at the `gos (typePrimRep orig_ty1) (typePrimRep orig_ty2)` and, seeing that the `orig_ty`s are unary, enter `go`. We will then split the applications such that, {{{ tc1, tc2 = (->) tc_args1, tc_args2 = ['LiftedPtrRep, 'LiftedPtrRep, a, IO ()] }}} Seeing that the tycons are equal, we will enter `gos` on each of the `tc_args`. Of course, one would think that `typePrimRep 'LiftedPtrRep` should throw an error, but because `gos` only forces the spine of the list, that error doesn't get thrown. Instead we incorrectly conclude that `'LiftedPtrRep` is a unary type and recurse into `go orig_ty1 orig_ty2`, thus looping. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13941> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 7

[GHC] #14117: stg-lint fails on unlifted-type join point binding
by GHC 28 Nov '17

28 Nov '17

#14117: stg-lint fails on unlifted-type join point binding -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.2.1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- While validating GHC with `BuildFlavour=devel2` and `-dstg-lint` I found that the linter chokes when it finds a join point of unlifted type (like this one from the `Binary` module), {{{ ghc-stage1: panic! (the 'impossible' happened) (GHC version 8.3.20170815 for x86_64-unknown-linux): *** Stg Lint ErrMsgs: in Stg2Stg *** <no location info>: warning: [RHS of $j_sDSL :: (# State# RealWorld, () #)] Let(rec) binder ‘$j_sDSL’ has unlifted type ‘(# State# RealWorld, () #)’ RHS: \r [] src<compiler/utils/Binary.hs:516:9-17> src<compiler/utils/Binary.hs:(516,21)-(518,26)> src<compiler/utils/Binary.hs:260:1-45> src<compiler/utils/Binary.hs:(239,1)-(246,34)> case readIntArray# [ww_sDSF 0# w1_sDSJ] of { ... }}} If I'm not mistaken there is no problem with join points having unlifted type (see `Note [CoreSyn let/app invariant]`). We just need to update the STG linter appropriately. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14117> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 4

[GHC] #14118: stg2stg passes appear to produce invalid STG
by GHC 28 Nov '17

28 Nov '17

#14118: stg2stg passes appear to produce invalid STG -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.2.1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- While building with GHC with `-dstg-lint -g3 -O0` (after fixing #14116 and #14117) I encountered a rather peculiar error, {{{ "inplace/bin/ghc-stage1" -hisuf hi -osuf o -hcsuf hc -static -O -H64m -Wall -this-unit-id integer-gmp-1.0.1.0 -hide-all-packages -i -ilibraries/integer-gmp/src/ -ilibraries/integer-gmp/dist-install/build -Ilibraries/integer-gmp/dist-install/build -ilibraries/integer-gmp/dist- install/build/./autogen -Ilibraries/integer-gmp/dist- install/build/./autogen -Ilibraries/integer-gmp/include -optP-include -optPlibrari es/integer-gmp/dist-install/build/./autogen/cabal_macros.h -package-id ghc-prim-0.5.1.0 -this-unit-id integer-gmp -Wall -XHaskell2010 -O -dcore- lint -g3 -ddump-to-file -ddump-stg -dcore-lint -dstg-lint -dcmm-lin t -no-user-package-db -rtsopts -Wno-deprecated-flags -Wnoncanonical- monad-instances -odir libraries/integer-gmp/dist-install/build -hidir libraries/integer-gmp/dist-install/build -stubdir libraries/intege r-gmp/dist-install/build -dynamic-too -c libraries/integer- gmp/src//GHC/Integer/Type.hs -o libraries/integer-gmp/dist- install/build/GHC/Integer/Type.o -dyno libraries/integer-gmp/dist- install/build/GHC/Integer /Type.dyn_o ghc-stage1: panic! (the 'impossible' happened) (GHC version 8.3.20170815 for x86_64-unknown-linux): *** Stg Lint ErrMsgs: in Stg2Stg *** <no location info>: warning: [in body of lambda with binders m0_scBy :: State# s_a2Em -> State# s_a2Em, s1_scBz :: State# s_a2Em] s'_scBA is out of scope <no location info>: warning: [in body of lambda with binders wild1_sdUv :: Int#] qr_sdUp is out of scope }}} Looking at the STG it appears that these warnings are absolutely correct, {{{#!hs svoid [InlPrag=INLINE (sat-args=1)] :: forall s. (State# s -> State# s) -> S s () [GblId, Arity=2, Caf=NoCafRefs, Str=<C(S),1*C1(U)><S,U>, Unf=OtherCon []] = \r [m0_scBy s1_scBz] src<libraries/integer-gmp/src/GHC/Integer/Type.hs:1957:1-48> case m0_scBy s1_scBz of s'_scBA { __DEFAULT -> src<libraries/integer- gmp/src/GHC/Integer/Type.hs:1957:37-48> (#,#) [s'_scBA ()]; }; }}} This is quite strange given that `s'_scBA` is clearly in scope, being bound as the case binder. We had the following from Core Prep, {{{#!hs -- RHS size: {terms: 10, types: 17, coercions: 0, joins: 0/0} GHC.Integer.Type.svoid [InlPrag=INLINE (sat-args=1)] :: forall s. (GHC.Prim.State# s -> GHC.Prim.State# s) -> GHC.Integer.Type.S s () [GblId, Arity=2, Caf=NoCafRefs, Str=<C(S),1*C1(U)><S,U>, Unf=OtherCon []] GHC.Integer.Type.svoid = \ (@ s_a2Em) (m0_scBy [Occ=Once!] :: GHC.Prim.State# s_a2Em -> GHC.Prim.State# s_a2Em) (s1_scBz [Occ=Once] :: GHC.Prim.State# s_a2Em) -> src<libraries/integer-gmp/src/GHC/Integer/Type.hs:1957:1-48> case m0_scBy s1_scBz of s'_scBA { __DEFAULT -> src<libraries/integer-gmp/src/GHC/Integer/Type.hs:1957:37-48> (# s'_scBA, GHC.Tuple.() #) } }}} The reason the linter fails here is due to the following logic in `lintStgExpr`, {{{#!hs in_scope <- MaybeT $ liftM Just $ case alts_type of AlgAlt tc -> check_bndr (tyConPrimRep tc) >> return True PrimAlt rep -> check_bndr [rep] >> return True MultiValAlt _ -> return False -- Binder is always dead in this case PolyAlt -> return True MaybeT $ addInScopeVars [bndr | in_scope] $ lintStgAlts alts scrut_ty }}} In the `svoid` case above we hit `MultiValAlt` path, which causes us to ignore the case binder. The fact that we hit `MultiValAlt` at all is a bit surprising given that the result is not an unboxed sum or tuple. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14118> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 11

[GHC] #14120: stg-lint is hopelessly broken
by GHC 28 Nov '17

28 Nov '17

#14120: stg-lint is hopelessly broken -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.2.1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- Today I spent a fair amount of time looking at the STG linter, having encountered a number of times in the past six months where I noticed it broke. While I thought fixing it would just be a few small tweaks, it seems that with every issue I fix another bug rears its ugly head. So far I have encountered and fixed, * #14116 * #14117 * #14118 The solutions to each of these has seemed rather obvious. However, now I seem to have run into a bit of a more fundamental issue: Consider this excerpt extracted from `Foreign.Storable`, {{{#!hs {-# LANGUAGE BangPatterns #-} module Hi where import GHC.Word import GHC.Ptr import GHC.Base import GHC.Num import Data.Bits import GHC.Fingerprint.Type peekFingerprint :: Ptr Fingerprint -> IO Fingerprint peekFingerprint p0 = do let peekW64 :: Ptr Word8 -> Int -> Word64 -> IO Word64 peekW64 _ 0 !i = return i peekW64 !p !n !i = peekW64 (p `plusPtr` 1) (n-1) (i `shiftL` 8) high <- peekW64 (castPtr p0) 8 0 low <- peekW64 (castPtr p0 `plusPtr` 8) 8 0 return (Fingerprint high low) }}} In particular notice the `castPtr` application. This triggers the STG linter with, {{{ ghc-stage1: panic! (the 'impossible' happened) (GHC version 8.3.20170815 for x86_64-unknown-linux): *** Stg Lint ErrMsgs: in Stg2Stg *** <no location info>: warning: [in body of lambda with binders p0_s2zB :: Ptr Fingerprint, eta_s2zC :: State# RealWorld] In a function application, function type doesn't match arg types: Function type: Ptr Word8 -> Int# -> Word# -> State# RealWorld -> (# State# RealWorld, Word64 #) Arg types: Ptr Fingerprint Int# Word# State# RealWorld Expression: $wpeekW64 p0_s2zB 8# 0## eta_s2zC }}} This is because by the time we are in Core Prep the `castPtr` is turned into a cast, which we discard in STG. Consequently, it seems that the comment attached to `stgEqType`, {{{#!hs stgEqType :: Type -> Type -> Bool -- Compare types, but crudely because we have discarded -- both casts and type applications, so types might look -- different but be the same. So reply "True" if in doubt. -- "False" means that the types are definitely different. -- -- Fundamentally this is a losing battle because of unsafeCoerce }}} is quite an understatement. Rather, there are exceedingly few cases where we can catch any type errors in STG. I think the only case which we can reliably catch is that of two types with explicitly different primreps. It's not clear what we can/should do about this. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14120> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 9

[GHC] #14116: Core lint error while compiling master
by GHC 28 Nov '17

28 Nov '17

#14116: Core lint error while compiling master -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 8.2.1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- While compiling e054c5f06451def4437d9d770ae156f034796c59 with the following `build.mk`, {{{ BuildFlavour = devel2 include mk/flavours/$(BuildFlavour).mk GhcStage2HcOpts += -eventlog GhcStage1HcOpts += -eventlog GhcLibHcOpts += -g3 -ddump-to-file -ddump-stg -dcore-lint -dstg-lint -dcmm-lint GhcRtsHcOpts += -g3 GhcStage2HcOpts += -g3 -ddump-to-file -ddump-stg -dcore-lint -dstg-lint -dcmm-lint }}} I encountered this Core lint failure while compiling `PprColour`, {{{ stgEqType: unequal PprColour [Char] ghc-stage1: panic! (the 'impossible' happened) (GHC version 8.3.20170815 for x86_64-unknown-linux): *** Stg Lint ErrMsgs: in Stg2Stg *** compiler/utils/PprColour.hs:62:1: warning: [RHS of defaultScheme :: Scheme] In a RHS constructor application, con type doesn't match arg types: Constructor type: PprColour -> PprColour -> PprColour -> PprColour -> PprColour -> PprColour -> Scheme Arg types: [Char] PprColour String String String String *** Offending Program *** ... }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14116> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 6

[GHC] #13110: GHC API allocates memory which is never GC'd
by GHC 27 Nov '17

27 Nov '17

#13110: GHC API allocates memory which is never GC'd -------------------------------------+------------------------------------- Reporter: DanielG | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: GHC API | Version: 8.0.1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- I've run into a very strange sort of memory leak it seems. GHC seems to manage to allocate memory which I cannot get rid of no matter what I try even after all references have disappeared. The allocated memory doesn't show up in heap profiles but is still visibly allocated (as seen in ps or pmap). I discovered this while investigating reports of large memory usage in ghc-mod where this issue is forcing us to dump completion information in a separate process to avoid the allocated memory sticking around for the rest of the user's session. The attached test case demonstrates this problem by first calling `getModuleInfo` for all modules in all visible packages and then just looping forever in `main`. I would expect all memory GHC allocated to be GC'd at this point but this does not happen. When run as {{{ $ ./Leaky -hide-all-packages }}} the test case consumes around 30M of memory on my system, if we however load some large package, say GHC itself {{{ $ ./Leaky -hide-all-packages -package ghc }}} the program will consume around 2-300M of memory which are never deallocated. The behaviour is not related to the loaded package though I tried it with `Cabal` too with the same result though smaller memory usage. At first I thought this might be related to large CAFs with IORefs inside not being GCd for some reason so I tried calling `resetCAFs` from the RTS before entering the loop. This however makes no difference whatsoever. I also tried forcing a major GC just to be save to no avail. Next I speculated that maybe the allocated memory is beyond the GC's control (malloc()ed or something) but looking at the memory map of the process with `pmap $(pgrep Leaky)` that the majority of the memory usage I'm seeing comes from address `0x0000000200000000` which is where the RTS allocates memory AFAIK. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13110> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 20

[GHC] #14262: fdReady cannot wait more than 49 days
by GHC 24 Nov '17

24 Nov '17

#14262: fdReady cannot wait more than 49 days -------------------------------------+------------------------------------- Reporter: nh2 | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.2.1 Keywords: | Operating System: Unknown/Multiple Architecture: x86_64 | Type of failure: None/Unknown (amd64) | Test Case: | Blocked By: Blocking: | Related Tickets: #8684, #13497 Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- {{{ import System.IO hWaitForInput stdin 4294968296 }}} This code waits for 1 second instead of 49.something days. The culprit: {{{ hWaitForInput :: Handle -> Int -> IO Bool fdReady(..., int msecs, ...) }}} Haskell `Int` is usally 64 bits. C `int` is usually 32 bits. Called here: {{{ ready :: FD -> Bool -> Int -> IO Bool ready fd write msecs = do r <- throwErrnoIfMinus1Retry "GHC.IO.FD.ready" $ fdReady (fdFD fd) (fromIntegral $ fromEnum $ write) (fromIntegral msecs) foreign import ccall safe "fdReady" fdReady :: CInt -> CInt -> CInt -> CInt -> IO CInt }}} We `fromIntegral` `Int` to `CInt` (== `Int32`), overflowing. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14262> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 7

[GHC] #13157: Opportunity to improve case-of-case
by GHC 24 Nov '17

24 Nov '17

#13157: Opportunity to improve case-of-case -------------------------------------+------------------------------------- Reporter: simonpj | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- Consider this code {{{ f g x = (case g x of True -> not False -> id) `seq` True }}} With GHC 8 (and -O) we get this Core {{{ f = \ (@ t_a1rk) (g_aqD :: t -> Bool) (x_aqE :: t) -> case case g_aqD x_aqE of { False -> id @ Bool; True -> not } of { __DEFAULT -> GHC.Types.True } }}} This is obviously bad. The reasoning is in the long, complicated `Note [Single-alternative cases]` in `Simplify.hs`. Happily with join points the entire Note becomes unnecessary; we can delete the special case that this Note describes, and the Note itself. Result: simpler compiler, and more optimal code. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13157> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 2