March 2018 - ghc-devs

Re: Memory usage exploding for complex pattern matching
by Victor Miraldo (UU) 29 Mar '18

29 Mar '18

Hello Ben, >> We came up with a "minimal" example that at least is self contained and >> triggers the problem; `runghc Minimal.hs` should finish with about 5GB of RAM. >> > Hmm, with 8.2 I'm seeing a peak memory usage of 3.5 GB, with that peak > occurring during renaming, not desugaring (when exhaustiveness checking > happens). Are you certain that the exhaustiveness checker is the > responsible party? Not at all! In fact, I ran some comparisons with "-ddump-phases" and plotted those. I'm attaching the graphs here. From base to "added E" we added 16 more pattern matches, then each "added" phase is an additional 16 pattern matches. The simplifier stands out there. I couldn't run a bigger case as I had hit the ceiling of my machine. Cheers, Victor

3 2

Re: Memory usage exploding for complex pattern matching
by Victor Miraldo (UU) 29 Mar '18

29 Mar '18

Hello! > You can turn off the pattern match checker, is that an option? How? > Can you post your code which shows the poor behaviour. It could be > good to profile that part of GHC to see if there are any easy gains > but it is known to be slow. I attached a minimal example in the original e-mail. The code that triggers the really bad case is not much bigger. It is divided in many more modules, though. Cheers, Victor

1 0

Memory usage exploding for complex pattern matching
by Victor Miraldo (UU) 29 Mar '18

29 Mar '18

Dear ghc-devs, We are working on a library and came across quite high memory consumption when compiling some code (generated by TH). This memory issue seems to come from the exhaustiveness checker as we found in: https://ghc.haskell.org/trac/ghc/ticket/11163 https://ghc.haskell.org/trac/ghc/ticket/11195 We came up with a "minimal" example that at least is self contained and triggers the problem; `runghc Minimal.hs` should finish with about 5GB of RAM. Our actual non-minimal scenario runs out of memory on a 16GB RAM machine. We tried with ghc 8.4.2, 8.0.2 and 7.10.3 and ghc-HEAD; all of those showed a similar problem, although the older ghc's used less memory. Do you have any suggestions to make our code finish compiling without running out of memory? Thanks!! Victor

3 2

Question about indirectees of BLACKHOLE closures
by Ömer Sinan Ağacan 29 Mar '18

29 Mar '18

Hi, I've been looking at BLACKHOLE closures and how the indirectee field is used and I have a few questions: Looking at evacuate for BLACKHOLE closures: case BLACKHOLE: { StgClosure *r; const StgInfoTable *i; r = ((StgInd*)q)->indirectee; if (GET_CLOSURE_TAG(r) == 0) { i = r->header.info; if (IS_FORWARDING_PTR(i)) { r = (StgClosure *)UN_FORWARDING_PTR(i); i = r->header.info; } if (i == &stg_TSO_info || i == &stg_WHITEHOLE_info || i == &stg_BLOCKING_QUEUE_CLEAN_info || i == &stg_BLOCKING_QUEUE_DIRTY_info) { copy(p,info,q,sizeofW(StgInd),gen_no); return; } ASSERT(i != &stg_IND_info); } q = r; *p = r; goto loop; } It seems like indirectee can be a TSO, WHITEHOLE, BLOCKING_QUEUE_CLEAN, BLOCKING_QUEUE_DIRTY, and it can't be IND. I'm wondering what does it mean for a BLACKHOLE to point to a - TSO - WHITEHOLE - BLOCKING_QUEUE_CLEAN - BLOCKING_QUEUE_DIRTY Is this documented somewhere or otherwise could someone give a few pointers on where to look in the code? Secondly, I also looked at the BLACKHOLE entry code, and it seems like it has a different assumption about what can indirectee field point to: INFO_TABLE(stg_BLACKHOLE,1,0,BLACKHOLE,"BLACKHOLE","BLACKHOLE") (P_ node) { W_ r, info, owner, bd; P_ p, bq, msg; TICK_ENT_DYN_IND(); /* tick */ retry: p = StgInd_indirectee(node); if (GETTAG(p) != 0) { return (p); } info = StgHeader_info(p); if (info == stg_IND_info) { // This could happen, if e.g. we got a BLOCKING_QUEUE that has // just been replaced with an IND by another thread in // wakeBlockingQueue(). goto retry; } if (info == stg_TSO_info || info == stg_BLOCKING_QUEUE_CLEAN_info || info == stg_BLOCKING_QUEUE_DIRTY_info) { ("ptr" msg) = ccall allocate(MyCapability() "ptr", BYTES_TO_WDS(SIZEOF_MessageBlackHole)); SET_HDR(msg, stg_MSG_BLACKHOLE_info, CCS_SYSTEM); MessageBlackHole_tso(msg) = CurrentTSO; MessageBlackHole_bh(msg) = node; (r) = ccall messageBlackHole(MyCapability() "ptr", msg "ptr"); if (r == 0) { goto retry; } else { StgTSO_why_blocked(CurrentTSO) = BlockedOnBlackHole::I16; StgTSO_block_info(CurrentTSO) = msg; jump stg_block_blackhole(node); } } else { ENTER(p); } } The difference is, when the tag of indirectee is 0, evacuate assumes that indirectee can't point to an IND, but BLACKHOLE entry code thinks it's possible and there's even a comment about why. (I don't understand the comment yet) I'm wondering if this code is correct, and why. Again any pointers would be appreciated. Thanks, Ömer

5 9

Cabal trouble
by Simon Peyton Jones 28 Mar '18

28 Mar '18

Trying to build the repro case in Trac #14208, I get this cabal install --with-ghc=/home/simonpj/5builds/ghc-8.4-branch/inplace/bin/ghc-stage2 Warning: cannot determine version of /home/simonpj/.cabal/bin/c2hs : "" Resolving dependencies... cabal: Could not resolve dependencies: trying: exceptions-0.8.3 (dependency of streamly) next goal: template-haskell (dependency of exceptions) rejecting: template-haskell-2.13.0.0/installed-2.1... (conflict: exceptions => template-haskell>=2.2 && <2.13) rejecting: template-haskell-2.12.0.0, template-haskell-2.11.1.0, template-haskell-2.11.0.0, template-haskell-2.10.0.0, template-haskell-2.9.0.0, template-haskell-2.8.0.0, template-haskell-2.7.0.0, template-haskell-2.6.0.0, template-haskell-2.5.0.0, template-haskell-2.4.0.1, template-haskell-2.4.0.0, template-haskell-2.3.0.1, template-haskell-2.3.0.0, template-haskell-2.2.0.0 (constraint from non-upgradeable package requires installed instance) After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: template-haskell, exceptions, streamly, transformers, base This is with my in-place build of GHC-8.4 (in-place but not modified in any way). I've done a 'cabal upate'. What am I doing wrong? Thanks Simon

2 1

RE: Missed beta reductions
by Simon Peyton Jones 28 Mar '18

28 Mar '18

Yes, it’s possible that he sequence you are seeing is what is happening to you. But why is that not what you want to see? What are you trying to achieve? Since this function might be applied to many different arguments, it’s probably not a good idea to do anything unconditionally… S From: David Feuer <david(a)well-typed.com> Sent: 27 March 2018 01:04 To: Simon Peyton Jones <simonpj(a)microsoft.com>; ghc-devs(a)haskell.org Subject: RE: Missed beta reductions Hmmm. So you think maybe it's doing something like (roughly; I don't have the exact case in front of me) (\p q -> f p (g p q)) this that ==> beta reduce let p = this q = that in f p (g p q) ==> inline q let p = this in f p (g p that) ? I tried marking the "this" in question INLINE CONLIKE [0]. Shouldn't that tell GHC that duplicating it is fine? I really want to see it for RULES. David Feuer Well-Typed, LLP -------- Original message -------- From: Simon Peyton Jones <simonpj(a)microsoft.com<mailto:simonpj@microsoft.com>> Date: 3/26/18 7:14 PM (GMT-05:00) To: David Feuer <david(a)well-typed.com<mailto:david@well-typed.com>>, ghc-devs(a)haskell.org<mailto:ghc-devs@haskell.org> Subject: RE: Missed beta reductions GHC always beta-reduces. It does not always inline. For that: -ddump-inlinings Simon From: ghc-devs <ghc-devs-bounces(a)haskell.org<mailto:ghc-devs-bounces@haskell.org>> On Behalf Of David Feuer Sent: 26 March 2018 23:59 To: ghc-devs(a)haskell.org<mailto:ghc-devs@haskell.org> Subject: Missed beta reductions What's the best way to find spots where GHC saw a redex and choose not to beta reduce? Is there a flag for that? It could be useful when trying to figure out why rules aren't firing. David Feuer Well-Typed, LLP

2 2

Missed beta reductions
by David Feuer 26 Mar '18

26 Mar '18

What's the best way to find spots where GHC saw a redex and choose not to beta reduce? Is there a flag for that? It could be useful when trying to figure out why rules aren't firing. David FeuerWell-Typed, LLP

2 2

Updating Phabricator
by Ben Gamari 26 Mar '18

26 Mar '18

Hello everyone, In around 90 minutes I will start a Phabricator upgrade. It has been a few months since the last update so it may take up 30 minutes or so. I'll send a message before starting. Cheers, - Ben

1 2

Windows
by Simon Peyton Jones 26 Mar '18

26 Mar '18

I've just got a new laptop (the old one's hard disk died) so I'm trying to get to the point of being able to build GHC again. I'm currently stuck here: ./configure configure: loading site script /usr/local/etc/config.site checking for gfind... no checking for find... /usr/bin/find checking for sort... /usr/bin/sort checking for GHC version date... inferred 8.5.20180323 checking for GHC Git commit id... inferred affdea82bb70e5a912b679a169c6e9a230e4c93e checking for ghc... /c/fp/HP-8.2.2/bin/ghc checking version of ghc... 8.2.2 GHC path canonicalised to: c:/fp/HP-8.2.2/bin/ghc checking build system type... x86_64-pc-msys checking host system type... x86_64-pc-msys checking target system type... x86_64-pc-msys Unknown OS msys Any ideas? Thanks Simon

6 12

Curious demand in a function parameter
by Ömer Sinan Ağacan 26 Mar '18

26 Mar '18

Hi, In this program {-# LANGUAGE MagicHash #-} module Lib where import Control.Exception import GHC.Exts import GHC.IO data Err = Err deriving (Show) instance Exception Err f :: Int -> Int -> IO Int f x y | x > 0 = IO (raiseIO# (toException Err)) | y > 0 = return 1 | otherwise = return 2 when I compile this with 8.4 -O2 I get a strict demand on `y`: f :: Int -> Int -> IO Int [GblId, Arity=3, Str=<S(S),1*U(U)><S(S),1*U(U)><S,U>, ...] but clearly `y` is not used on all code paths, so I don't understand why we have a strict demand here. I found this example in the comments around `raiseIO#`: -- raiseIO# needs to be a primop, because exceptions in the IO monad -- must be *precise* - we don't want the strictness analyser turning -- one kind of bottom into another, as it is allowed to do in pure code. -- -- But we *do* want to know that it returns bottom after -- being applied to two arguments, so that this function is strict in y -- f x y | x>0 = raiseIO blah -- | y>0 = return 1 -- | otherwise = return 2 However it doesn't explain why we want be strict on `y`. Interestingly, when I try to make GHC generate a worker and a wrapper for this function to make the program fail by evaluating `y` eagerly I somehow got a lazy demand on `y`: {-# LANGUAGE MagicHash #-} module Main where import Control.Exception import GHC.Exts import GHC.IO data Err = Err deriving (Show) instance Exception Err f :: Int -> Int -> IO Int f x y | x > 0 = IO (raiseIO# (toException Err)) | y > 0 = f x (y - 1) | otherwise = f (x - 1) y main = f 1 undefined I was thinking that this program should fail with "undefined" instead of "Err", but the demand I got for `f` became: f :: Int -> Int -> IO Int [GblId, Arity=2, Str=<S(S),1*U(U)><L,1*U(U)>, ...] which makes sense to me. But I don't understand how my changes can change `y`s demand, and why the original demand is strict rather than lazy. Could anyone give me some pointers? Thanks Ömer

3 2