- ghc-devs - Haskell.org

Selector thunks again
by David Feuer 09 Mar '20

09 Mar '20

The fragility of this feature remains frustrating. A few days ago, I wrote this code for building a complete binary tree from its breadth-first traversal. (This is an improvement of a version by Will Ness.) data Tree a = Empty | Node a (Tree a) (Tree a) deriving Show -- An infinite list data IL a = a :< IL a infixr 5 :< bft :: [a] -> Tree a bft xs = tree where tree :< subtrees = go xs subtrees go :: [a] -> IL (Tree a) -> IL (Tree a) go (a : as) ~(b1 :< ~(b2 :< bs)) = Node a b1 b2 :< go as bs go [] _ = fix (Empty :<) When GHC compiles the lazy patterns, we get something essentially like this: go (a : as) ys = Node a (case ys of b1 :< _ -> b1) (case ys of _ :< b2 :< _ -> b2) :< go as (case ys of _ :< _ :< bs -> bs) Now `case ys of b1 :< _ -> b1` is a selector thunk, which is cool. The GC can reduce it as soon as either of the other thunks is forced. But neither of the other two case expressions is a selector thunk, so neither will ever be reduced by the GC. If I consume the result tree using an inorder traversal, for example, then all the elements in the left subtree of the root will remain live until I start to consume the right subtree of the root. I can instead write this: go (a : as) ys = Node a b1 b2 :< go as bs where {-# NOINLINE b2bs #-} b1 :< b2bs = ys b2 :< bs = b2bs Now all the suspended selections are selector thunks, so things should clean up nicely. There are still three problems, though. The first is that this is harder to read. The second is that now we have four suspended selections instead of three. Finally, if b1 is not the first one forced, we'll need to force two thunks instead of one. Can't we do any better?

2 1

Class op rules
by Christiaan Baaij 09 Mar '20

09 Mar '20

Hello, The other day I was experimenting with RULES and got this warning: src/Clash/Sized/Vector.hs:2159:11: warning: [-Winline-rule-shadowing] Rule "map Pack" may never fire because rule "Class op pack" for ‘pack’ might fire first Probable fix: add phase [n] or [~n] to the competing rule | 2159 | {-# RULES "map Pack" map pack = id #-} The warning seems to suggests two things: 1. "Class op" -> "dictionary projection" are implemented as rewrite rules and executed the same way as other user-defined RULES 2. These rules run first, and you cannot run anything before them Now my question is, is 1. actually true? or is that warning just a (white) lie? If 1. is actually true, would there be any objections to adding a "-1" phase: where RULES specified to start from phase "-1" onward fire before any of the Class op rules. I'm quite willing to implement the above if A) Class op rules are actually implemented as builtin RULES; B) there a no objections to this "-1" phase. Thanks, Christiaan

5 7

Typechecker plugin proposal, ticket #15147
by Nicolas Frisby 06 Mar '20

06 Mar '20

If you know of any typechecker plugin authors I've missed, please add them to the thread. In the comments of ticket https://gitlab.haskell.org/ghc/ghc/issues/15147 several of us agreed that the behavior of the typechecker plugin interface should change: GHC should no longer unflatten the fmvs in the Wanteds before passing them to the plugin. This is presumably a breaking change to plugins. We might be able to get by with some sort of flag indicating if the plugin expects the Wanteds flattened or not, but ideally GHC would just always pass them flattened. I'm unaware of any established policy about interface changes at this level, whether we've somehow committed to backwards-compatibility here or not. Anyone know? Plugin authors: would you look over the ticket comments and share your thoughts here? We're looking to build some sort of consensus about how to proceed without shocking the API users. Thank you for your time. -Nick

2 1

Number of threads in haskell program
by chessai . 04 Mar '20

04 Mar '20

Hi devs, Recently I became interested in obtaining the rough number of (non-GC) threads that are alive in a Haskell program. Intuitively this seemed like something that the RTS would expose in some way - but I couldn't find any such exposition in base. I then thought that I could simply total the size of each run_queue_hd in each capability, but the machinery for doing that doesn't seem to be exposed. Any thoughts? Thanks

2 1

Blocking MVar# primops not performing stack checks?
by Shao, Cheng 04 Mar '20

04 Mar '20

Hi all, When an MVar# primop blocks, it jumps to a function in HeapStackCheck.cmm which pushes a RET_SMALL stack frame before returning to the scheduler (e.g. the takeMVar# primop jumps to stg_block_takemvar for stack adjustment). But these functions directly bump Sp without checking for possible stack overflow, I wonder if it is a bug? Cheers, Cheng

3 2

Feasibility of native RTS support for continuations?
by Alexis King 03 Mar '20

03 Mar '20

Hi all, tl;dr: I want to try to implement native support for capturing slices of the RTS stack as a personal experiment; please tell me the obstacles I am likely to run into. Much more context follows. --- I have been working on an implementation of an algebraic effect system that uses unsafe primops to be as performant as possible. However, the unavoidable need to CPS the entire program balloons heap allocation. Consider the following definition: f a b = g a >>= \c -> h (b + c) Assume `g` and `h` are not inlined. If the monad used is IO, this will be compiled efficiently: the result of `g` is returned on the stack, and no closure needs to be allocated for the lambda. However, if the monad supports capturing the continuation, the above definition must be CPS’d. After inlining, we end up with f a b = \k -> let lvl = \c -> h (b + c) k in g a lvl which must allocate a closure on the heap. This is frustrating, as it must happen for every call to a non-inlined monadic operation, even if that operation never captures the continuation. In an algebraic effect system, there are many shortcuts that avoid the need to capture the continuation, and under my implementation, large swaths of code never do so. I’ve managed to exploit that to get some savings, but I can’t escape the need to allocate all these closures. This motivates my question: how difficult would it be to allow capturing a portion of the RTS call stack directly? My requirements are fairly minimal, as continuations go: 1. Capturing a continuation is only legal from within a strict state thread (i.e. IO or strict ST). 2. The continuation is captured up to a prompt, which would be a new kind of RTS stack frame. Prompts are not tagged, so there is only ever exactly one prompt active at any time (which may be the root prompt). 3. Capturing a continuation is unsafe. The behavior of capturing a continuation is undefined if the current prompt was not created by the current state thread (and it is never legal to capture up to the root prompt). 4. Applying a continuation is unsafe. Captured continuations return `Any`, and type safety is the caller’s obligation. 5. Continuations are “functional,” which is to say applying them does not trigger any additional stack unwinding. This minimal support for primitive continuation capturing would be enough to support my efficient, safe delimited control implementation. In my ignorant mind, implementing this ought to be as simple as defining two new primops, reset# :: (State# s -> (# State# s, a #)) -> State# s -> (# State# s, a #) shift# :: ((a -> State# s -> (# State# s, Any #)) -> State# s -> (# State# s, Any #)) -> State# s -> (# State# s, a #) where reset# pushes a new prompt frame and shift# captures a slice of the RTS stack up to that frame and copies it into the heap. Restoring a continuation would copy all the captured frames onto the end of the current stack. Sounds simple enough! I would like to experiment with implementing something like this myself, just to see if it would really work, but somehow I doubt it is actually as simple as it sounds. Minor complications are fine, but what worries me are major obstacles I haven’t found in my limited attempts to learn about the RTS. So far, I’ve read the old “The New GHC/Hugs Runtime System” paper, which still seems mostly accurate from a high level, though I imagine many details have changed since then. I’ve also read the Commentary/RTS/Storage/Stack wiki page, and I’ve peeked at some of the RTS source code. I’ve also skimmed a handful of other papers and wiki pages, but it’s hard to know what I’m looking for. Can anyone point me to better resources or give me some insight into what will be hard? Thanks in advance, Alexis

4 15

Reason for skipping sanity checking threads and mut_lists before a GC?
by Ömer Sinan Ağacan 03 Mar '20

03 Mar '20

Hi, With `+RTS -DS` we call this function before and after a GC: void checkSanity (bool after_gc, bool major_gc) { checkFullHeap(after_gc && major_gc); checkFreeListSanity(); // always check the stacks in threaded mode, because checkHeap() // does nothing in this case. if (after_gc) { checkMutableLists(); checkGlobalTSOList(true); } } For some reason this skips mut lists and threads before a GC, I don't understand why that is necessary. Does anyone know the reason? Thanks, Ömer

1 0

Re: [ANNOUNCE] GHC 8.8.3 is now available
by Ben Gamari 29 Feb '20

29 Feb '20

Ben Gamari <ben(a)well-typed.com> writes: > --text follows this line-- > <#part sign=pgpmime> > > Hello everyone, > > The GHC team is proud to announce the release of GHC 8.8.3. The source > distribution, binary distributions, and documentation are available at > > https://downloads.haskell.org/~ghc/8.8.3 > > Release notes are also available [1]. > Unfortunately there has been a slight mix-up in the binary distributions associated with this release and I will need to do a re-spin. I have pulled down the binary distributions for the time being. They should be back in a few hours. Many apologies for the confusion! Cheers, - Ben

2 2

hsinspect-0.0.11 released
by Tseen She 28 Feb '20

28 Feb '20

Hello ghc developers, With many thanks to Daniel Grober, I have released 0.0.11 of hsinspect to Hackage, which now supports ghc versions 8.4.x, 8.6.x and 8.8.x. I would like to bring this tool to the attention of this community of ghc developers because it is using the ghc api exclusively to provide semantic information about .hs files, so it may be of interest to you to see a tool using your api in the wild. And selfishly, I learnt so much from Daniel's reviews of two short functions recently (which were subject to a change in ghc behavour in 8.8) that I would very much welcome further reviews by this group. The source code is split across three parts: The plugin (an alternative to hie-bios, which didn't work well for me): https://gitlab.com/tseenshe/hsinspect/-/blob/master/plugin/GhcFlags/Plugin.… The library, one file per feature: https://gitlab.com/tseenshe/hsinspect/-/tree/master/library/HsInspect and the Executable (setup, then just call the relevant library): https://gitlab.com/tseenshe/hsinspect/-/blob/master/exe/Main.hs My usecase is to provide semantic information for my Emacs mode, which is a completely from-scratch haskell-mode that I've been working on in my free time for the past year: https://gitlab.com/tseenshe/haskell-tng.el For anybody curious about the wider project, there are some screenshots of the features in the README, which also fully justifies why I undertook this endeavour: i.e. why I don't believe I could achieve what I wanted in haskell-mode or LSP. I am content for this tool to be used just by myself, my work colleagues, and anybody who is interested in contributing. Best regards, Tseen She

1 0

8.8.x change in lookupModule / findModule semantics?
by Tseen She 28 Feb '20

28 Feb '20

Hi ghc-devs, When upgrading my dev tool to use the 8.8.x api I have found that calls to lookupModule / findModule are failing on uncompilable code, whereas these calls succeeded on 8.4 and 8.6. Unlike another question that I have asked [1] which seems related, I don't intend to do any typechecking here, I only want access to the ModuleInfo. Has this been an intentional change? Is there a way to recover the original behaviour? If I cannot get this to work then it means that a new feature introduced in https://gitlab.haskell.org/ghc/ghc/merge_requests/1541 that I was quite excited about is unusable :-( [1] https://mail.haskell.org/pipermail/ghc-devs/2020-February/018655.html

2 12