- ghc-tickets - Haskell.org

[GHC] #10359: Tuple constraint synonym led to asymptotic performance lossage
by GHC 18 May '15

18 May '15

#10359: Tuple constraint synonym led to asymptotic performance lossage -------------------------------------+------------------------------------- Reporter: axch | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 Keywords: | Operating System: Linux Architecture: x86_64 | Type of failure: Runtime (amd64) | performance bug Test Case: | Blocked By: Blocking: | Related Tickets: Differential Revisions: | -------------------------------------+------------------------------------- I encountered a situation where using a tuple constraint synonym produced an asymptotic runtime performance degradation, from linear to quadratic time. The enclosed Bug.hs reproduces the potentially relevant features of the problem. The function `test` there is quadratic, but should be linear. The file also contains a definition of `test2`, which differs only in eschewing the constraint synonym, and is linear, as it should be. In more detail: the program tries to count to 20,000 in a rather contrived way. It maintains a Box, which holds the current count and a function that can accept an increment and a count to produce a new count (namely, `(+)`). The function `do_step` lifts incrementation to Boxes, using their contained function to update their stored count. The first tricky thing is that the incrementing function stored in the Box is polymorphic in both arguments separately: it can accept increments that are not the same type as the stored count (and converts them). The second tricky thing is that I don't want to expose the type variable for the increment type (as opposed to the count type) to clients of the Box, so the Box's incrementing function is stored polymorphic (with an explicit forall). Finally, I want to impose the constraint `(Fractional num, Real num)` on allowable increments (even though this example only needs `Real num`). This constellation of desiderata conspires to produce quadratic performance: doubling the parameter to `test` (but not `test2`) multiplies its runtime by 4. Inspecting the heap profile indicates a growing accumulation of closures when running `test` (but not `test2`). Inspecting the generated stg (enclosed) locates a potential culprit: apparently when `do_step` (but not `do_step2`) reconstructs the Box, it changes the `func` stored there to be a fresh closure that destructures the `Numerical` constraint tuple, constructs a fresh one, and calls the function that was in that slot previously with it. I hypothesize that this accumulates as a chain that performs a linear number of such destructurings and restructurings at each increment, leading to the observed quadratic runtime and linear memory growth. Incidentally, I checked whether record wildcards were the issue (no: `test4`) and whether updating just the `obj` field solves it (yes: `test3`). Sadly, the latter solution is not directly usable in my actual application because of "Record update for insufficiently polymorphic field". -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10359> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 11

[GHC] #10228: Increased memory usage with GHC 7.10.1
by GHC 18 May '15

18 May '15

#10228: Increased memory usage with GHC 7.10.1 -------------------------------------+------------------------------------- Reporter: jstolarek | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: Compile-time Unknown/Multiple | performance bug Test Case: | Blocked By: Blocking: | Related Tickets: Differential Revisions: | -------------------------------------+------------------------------------- (Repost from ghc-devs) I just uprgaded both of my machines to use GHC 7.10.1. I keep sandboxed installations of GHC and this means I had to rebuild Agda and Idris because the binaries built with GHC 7.8.4 were stored inside deactivated 7.8.4 sandbox. Sadly, I had problems building both Agda and Idris due to GHC taking up all of available memory. With Idris the problematic module was Idris.ElabTerm (~2900LOC). The interesting part of the story is that when I do a clean build of Idris GHC consumes all of memory when compiling that module and I have to kill the build. But when I restart the build after killing GHC the module is compiled using a reasonable amount of memory and within reasonable time. With Agda the problematic module is Agda.TypeChecking.Serialise (~2000LOC). The trick with killing the build and restarting it didn't work in this case. I had to compile Agda with GHC 7.8.4 (which works without problems though the mentioned module still requires a lot of memory) and alter my setup so that Agda binary is not stored inside GHC sandbox. I wonder if any of you came across similar issues with GHC 7.10.1? Do we have any performance data that allows to compare memory usage and performance of GHC 7.10.1 with previous stable releases? All of the above happened on 64bit Debian Wheezy with 2GB of RAM. On my second machine, this time with 4GB of RAM, compiling Agda ran out of memory (again Agda.TypeChecking.Serialise module) and I had to kill the build. But once I restarted the build the module was compiled succesfully in a matter of minutes and using around 50% of memory. This looks like some kind of memory leak in GHC. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10228> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 6

[GHC] #10427: Template variable unbound in rewrite rule
by GHC 18 May '15

18 May '15

#10427: Template variable unbound in rewrite rule -------------------------------------+------------------------------------- Reporter: crockeea | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: Compile-time Unknown/Multiple | crash Test Case: | Blocked By: Blocking: | Related Tickets: Differential Revisions: | -------------------------------------+------------------------------------- This may be due to a bug in [singletons](https://hackage.haskell.org/package/singletons), but GHC requested I post a bug here, so I am dutifully complying. The following code fails with -O2 (but succeeds with -O1): {{{#!hs {-# LANGUAGE DataKinds, GADTs, KindSignatures, PolyKinds, ScopedTypeVariables, TemplateHaskell, TypeFamilies, UndecidableInstances #-} module Error where import qualified GHC.Num as Num import Data.Singletons.Prelude hiding (sMin, sMax, MinSym0, MaxSym0) import Data.Singletons.TH import Data.Type.Natural as N hiding ((:-)) singletons [d| newtype Foo = Bar Nat deriving (Eq,Show) |] singletons [d| ppMul :: Foo -> [Foo] -> [Foo] ppMul x@(Bar p) pps@((Bar p'):pps') = (Bar p'):ppMul x pps' |] }}} with the message {{{ ghc: panic! (the 'impossible' happened) (GHC version 7.10.1 for x86_64-unknown-linux): Template variable unbound in rewrite rule n_Xdkc [n_adis, n1_adix, n_adiA, sc_se5w, sg_se5x, sc_se5y] [n_Xdk2, n1_Xdk8, n_Xdkc, sc_Xe79, sg_Xe7b, sc_Xe7d] [TYPE Let_1627437169XSym3 n_adis n_adiA n1_adix, TYPE n1_adix, (SBar @ ('Bar n_adis) @ n_adis @~ <'Bar n_adis>_N sc_se5w) `cast` (sg_se5x :: R:SingFooz (BarSym1 n_adis) ~R# Sing (Apply BarSym0 n_adis)), sc_se5y] [TYPE Let_1627437169XSym3 n_adis n_XdjP n1_XdjH, TYPE n1_XdjH, (SBar @ ('Bar n_adis) @ n_adis @~ <'Bar n_adis>_N sc_se5w) `cast` (Sub (Sym TFCo:R:SingFooz[0]) (Sym (TFCo:R:ApplyFooNatBarSym0l0[0] <n_adis>_N)) :: R:SingFooz (BarSym1 n_adis) ~R# Sing (Apply BarSym0 n_adis)), sPps'_aciK] }}} I found that if I rewrite `ppmul` as `ppMul x pps@((Bar p'):pps') = (Bar p'):ppMul x pps'` (i.e. remove the pattern match on `x`), then ghc -02 succeeds. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10427> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 1

Re: [GHC] #7695: Hang when locale-archive and gconv-modules are not there
by GHC 17 May '15

17 May '15

#7695: Hang when locale-archive and gconv-modules are not there -------------------------------------+------------------------------------- Reporter: hpd | Owner: Type: bug | Status: new Priority: highest | Milestone: 7.10.2 Component: None | Version: 7.8.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime crash | Unknown/Multiple Blocked By: | Test Case: Related Tickets: #8977, #10298 | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by hsyl20): Fixed with my patch for #10298 (see my comment there). Can you check that you obtain the same result on your system with this code: {{{ $> cat test.c #include <langinfo.h> #include <locale.h> #include <stdio.h> int main() { setlocale(LC_CTYPE, ""); printf("LC_CTYPE: %s\n", nl_langinfo(CODESET)); return 0; } $> gcc -static -L/path/to/static/glibc test.c -o test $> sudo chroot `pwd` /test LC_CTYPE: ANSI_X3.4-1968 }}} I don't know if the default LC_CTYPE is the same everywhere... (You should get the same result if you have compiled the glibc with --disable-shared when you call ./test directly without the chroot) -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/7695#comment:18> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 0

Re: [GHC] #4012: Compilation results are not deterministic
by GHC 15 May '15

15 May '15

#4012: Compilation results are not deterministic -------------------------------------+------------------------------------- Reporter: kili | Owner: Fuuzetsu Type: bug | Status: new Priority: high | Milestone: 7.12.1 Component: Compiler | Version: 6.12.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Other | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by nomeata): I agree that this should not hold up a patch level release that others are waiting for. Better fix this properly, and then, maybe, see if it can be part of a 7.10.3. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/4012#comment:84> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 0

Re: [GHC] #4012: Compilation results are not deterministic
by GHC 15 May '15

15 May '15

#4012: Compilation results are not deterministic -------------------------------------+------------------------------------- Reporter: kili | Owner: Fuuzetsu Type: bug | Status: new Priority: high | Milestone: 7.12.1 Component: Compiler | Version: 6.12.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Other | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by simonpj): * priority: normal => high Comment: George Colpitts asks: the milestone is 7.12.1, should it be 7.10.2? It would be unusual to delay a patch-level release for a ticket that has been open for five years. Doing so would mean indefinitely delaying 7.10.2. By “indefinite” I don't mean "forever", I just mean “we don’t know how long”: * We don’t have anyone owning the ticket, * we don’t know how difficult it is to fix; and * a full bulletproof fix is probably quite challenging (e.g. forcing the unique-supply generator to give reproducible results even though some (ultimately irrelevant) bits of the environment have changes, would be difficult) At the same time we have some folk urging us to get 7.10.2 out sooner rather than later. So everyone: * If you think we should delay 7.10.2 pending at least some progress on this, please say so on the ticket. * Better still, roll up your sleeves and take a crack at understanding more precisely what is going on, and proposing a fix. Simon -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/4012#comment:83> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 0

[GHC] #10293: CallArity taking 20% of compile time
by GHC 15 May '15

15 May '15

#10293: CallArity taking 20% of compile time -------------------------------------+------------------------------------- Reporter: simonpj | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Blocked By: Test Case: | Related Tickets: Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- The `CallArity` analysis can, apparently, take 20% of compile time. That's a lot for one analysis that is on by default with `-O`. Michael Terepeta writes: Out of curiosity I had a look at compiling haskell-src-exts since that takes quite a while. I've used ghc HEAD and 7.8.4 (both built with BuildFlavour=prof & bootstrapped with a standard ghc 7.8.4) and it's interesting -- the current HEAD takes quite a bit longer and allocates way more than 7.8.4. One of the main things that stand out is the CallArity analysis (which IIRC was not there in 7.8.4). So unless I messed something up with measuring, the analysis seem to be pretty expensive. HEAD {{{ Sun Apr 12 15:52 2015 Time and Allocation Profiling Report (Final) ghc +RTS -p -RTS [...] total time = 147.84 secs (147841 ticks @ 1000 us, 1 processor) total alloc = 172,378,600,408 bytes (excludes profiling overheads) COST CENTRE MODULE %time %alloc SimplTopBinds SimplCore 32.4 28.8 CallArity SimplCore 18.4 25.6 lintAnnots CoreLint 4.5 4.6 CoreTidy HscMain 4.5 5.1 pprNativeCode AsmCodeGen 3.2 3.4 OccAnal SimplCore 3.2 3.1 occAnalBind.assoc OccurAnal 2.6 2.5 StgCmm HscMain 2.3 1.9 Simplify SimplCore 2.1 0.2 RegAlloc AsmCodeGen 2.1 2.4 FloatOutwards SimplCore 2.0 1.6 regLiveness AsmCodeGen 1.9 1.9 tc_rn_src_decls TcRnDriver 1.8 1.3 sink CmmPipeline 1.7 1.5 NewStranal SimplCore 1.3 1.5 genMachCode AsmCodeGen 1.1 1.0 layoutStack CmmPipeline 1.0 1.0 }}} HEAD with `-fno-call-arity` {{{ Sun Apr 12 18:16 2015 Time and Allocation Profiling Report (Final) ghc +RTS -p -RTS [...] -fno-call-arity total time = 113.71 secs (113714 ticks @ 1000 us, 1 processor) total alloc = 121,884,896,720 bytes (excludes profiling overheads) COST CENTRE MODULE %time %alloc SimplTopBinds SimplCore 37.2 36.6 CoreTidy HscMain 6.0 7.3 lintAnnots CoreLint 5.8 6.5 pprNativeCode AsmCodeGen 4.1 4.8 OccAnal SimplCore 3.6 3.8 occAnalBind.assoc OccurAnal 2.9 3.2 StgCmm HscMain 2.9 2.6 RegAlloc AsmCodeGen 2.6 3.4 FloatOutwards SimplCore 2.6 2.3 regLiveness AsmCodeGen 2.5 2.8 tc_rn_src_decls TcRnDriver 2.4 1.9 Simplify SimplCore 2.4 0.3 sink CmmPipeline 2.1 2.2 NewStranal SimplCore 1.7 2.1 genMachCode AsmCodeGen 1.4 1.4 layoutStack CmmPipeline 1.4 1.4 NativeCodeGen CodeOutput 1.1 1.2 FloatInwards SimplCore 1.1 1.4 do_block Hoopl.Dataflow 1.0 0.6 Digraph.scc Digraph 0.8 1.3 }}} GHC 7.8.4 {{{ Sun Apr 12 15:41 2015 Time and Allocation Profiling Report (Final) ghc +RTS -p -RTS [...] total time = 93.11 secs (93112 ticks @ 1000 us, 1 processor) total alloc = 103,135,975,120 bytes (excludes profiling overheads) COST CENTRE MODULE %time %alloc SimplTopBinds SimplCore 38.5 37.4 pprNativeCode AsmCodeGen 6.2 7.2 StgCmm HscMain 3.9 4.2 RegAlloc AsmCodeGen 3.7 5.1 occAnalBind.assoc OccurAnal 3.3 3.6 OccAnal SimplCore 3.3 3.6 regLiveness AsmCodeGen 3.1 3.4 FloatOutwards SimplCore 2.9 2.4 sink CmmPipeline 2.8 2.8 Simplify SimplCore 2.6 0.3 tc_rn_src_decls TcRnDriver 2.4 2.1 genMachCode AsmCodeGen 1.9 2.0 NewStranal SimplCore 1.8 2.1 layoutStack CmmPipeline 1.8 1.8 Core2Core HscMain 1.3 1.2 deSugar HscMain 1.1 1.1 do_block Hoopl.Dataflow 1.1 0.7 CoreTidy HscMain 1.0 1.1 CorePrep HscMain 1.0 1.1 Digraph.scc Digraph 0.9 1.5 versioninfo MkIface 0.9 1.0 zonkEvBndr_zonkTcTypeToType TcHsSyn 0.6 1.4 }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10293> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 24

Re: [GHC] #7398: RULES don't apply to a newtype constructor
by GHC 15 May '15

15 May '15

#7398: RULES don't apply to a newtype constructor -------------------------------------+------------------------------------- Reporter: shachaf | Owner: Type: bug | Status: new Priority: normal | Milestone: 7.12.1 Component: Compiler | Version: 7.6.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: #6082, #10418 | Differential Revisions: -------------------------------------+------------------------------------- Changes (by duncan): * related: #6082 => #6082, #10418 -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/7398#comment:8> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 0

[GHC] #10418: Incorrect RULE warning on constructor, and inablilty to {-# INLINE [0] #-} constrcutor
by GHC 15 May '15

15 May '15

#10418: Incorrect RULE warning on constructor, and inablilty to {-# INLINE [0] #-} constrcutor -------------------------------------+------------------------------------- Reporter: duncan | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Blocked By: Test Case: | Related Tickets: Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- This is mentioned in passing in ticket #10417, but pulling it out here to track it. Given a type {{{#!hs data Decoder s s' where Done :: Decoder s s Sequence :: Decoder a b -> Decoder b c -> Decoder a c }}} and rules: {{{#!hs {-# RULES "Sequence/Done" forall sa. Sequence Done sa = sa #-} {-# RULES "Sequence/Sequence" forall a b c. Sequence (Sequence a b) c = Sequence a (Sequence b c) #-} }}} GHC gives us warnings like: {{{ Rule "Sequence/Done" may never fire because ‘RulesExample.Sequence’ might inline first Probable fix: add an INLINE[n] or NOINLINE[n] pragma on ‘RulesExample.Sequence’ }}} but it's currently syntactically invalid to write such a pragma: {{{#!hs {-# INLINE [0] Done #-} {-# INLINE [0] Sequence #-} }}} Now there's two problems here: 1. This warning is wrong most of the time 2. The warning is actually right sometimes, but there's nothing you can do about it. For 1, most of the time constructors remain as themselves in core, with no wrappers to get in the way of rule matching. In these cases the warning is just wrong. For 2, for some constructors ghc does rewrite them. In this case the rewrite rule really will fail to match. But we cannot tell GHC to delay the rewriting of them because we cannot write INLINE pragmas for them. So for this case it would be useful to allow us to write INLINE pragmas. In the above example, GHC will rewrite `Done` during optimisation, e.g. an excerpt from ticket #10417: {{{ $WDone = \ @ s_anC -> Done @~ <s_anC>_N working_example2 = \ @ b_aw2 -> AlterStack (working_example3) ($WDone) }}} This can and will interfere with rule matching. In this case it would be desirable to be able to write {{{ {-# INLINE [0] Done #-} }}} Currently that's a syntax error. For the case of nullary constructors, a workaround is to consistently use a smart constructor with an `INLINE [0]` pragma, and do all the rule matching on that. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10418> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 2

[GHC] #10415: ForeignPtr touched in FFI wrapper is never discarded
by GHC 14 May '15

14 May '15

#10415: ForeignPtr touched in FFI wrapper is never discarded -------------------------------------+------------------------------------- Reporter: Deewiant | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.1 Keywords: | Operating System: Linux Architecture: x86_64 | Type of failure: Runtime (amd64) | performance bug Test Case: | Blocked By: Blocking: | Related Tickets: Differential Revisions: | -------------------------------------+------------------------------------- If a `ForeignPtr` is touched in an FFI wrapper (`foreign import ccall "wrapper"`), it appears to never be discarded. For example, the program below demonstrates a space leak due to multiple `mallocForeignPtrBytes` results hanging around forever. {{{#!hs import Control.Concurrent (threadDelay) import Foreign.ForeignPtr import Foreign.Marshal.Utils (fillBytes) import Foreign.Ptr (FunPtr) foreign import ccall "wrapper" wrap :: IO () -> IO (FunPtr (IO ())) main :: IO () main = flip mapM_ [0..50] $ \_ -> do let len = 30 * 2^20 fp <- mallocForeignPtrBytes len withForeignPtr fp $ \p -> fillBytes p 0 len _ <- wrap $ touchForeignPtr fp threadDelay 100000 }}} Each of the 50 iterations allocates 30 megabytes of memory via `mallocForeignPtrBytes`. None of those allocations is ever freed, resulting in around 1.5 gigabytes of memory use by the time the program terminates. (Neither `fillBytes` nor `threadDelay` are required to reproduce the issue: they are there to respectively touch the memory so that it's fully allocated by the OS and to slow the program down so that one can view the memory usage gradually increasing e.g. with `top`.) Example run, with almost 1.6 gigabytes of memory usage reported by GNU time (the "maxresident" bit): {{{ $ ghc --make asdf.hs [1 of 1] Compiling Main ( asdf.hs, asdf.o ) Linking asdf ... $ /usr/bin/time ./asdf 0.12user 0.54system 0:05.77elapsed 11%CPU (0avgtext+0avgdata 1587300maxresident)k 0inputs+0outputs (0major+138078minor)pagefaults 0swaps }}} People bitten by this can probably work around it without too much trouble by either: 1. Simply avoiding the trigger, i.e. `touchForeignPtr` in a wrapper. In my case, that amounts to changing some functions to take `Ptr a` instead of `ForeignPtr a` and moving some `withForeignPtr` calls around, possibly adding manual `touchForeignPtr` calls somewhere. 2. Switching to manual memory management with functions like `mallocBytes` and `free`. But the issue seems rather nasty regardless. (At least it took me a while to pin down.) Hopefully it can be fixed without too much trouble. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10415> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler

1 2