October 2019 - ghc-devs

Handling source locations in HsSyn via TTG
by Simon Peyton Jones 30 Oct '19

30 Oct '19

Friends As you know * We are trying to use "Trees That Grow<https://gitlab.haskell.org/ghc/ghc/wikis/implementing-trees-that-grow>" (TTG) to move HsSyn towards a situation in which GHC is merely a client of a generic HsSyn data type that can be used by clients other than GHC. * One sticking point has been the question of attaching source locations. We used to have a "ping-pong" style, in which very node is decorated with a source location, but that's a bit more awkward in TTG. * This wiki page<https://gitlab.haskell.org/ghc/ghc/wikis/implementing-trees-that-grow/handl…> outlines some choices, while ticket #15495<https://gitlab.haskell.org/ghc/ghc/issues/15495> has a lot of discussion. * HEAD embodies Solution A. But it has the disadvantage that the type system doesn't enforce locations to be present at all. That has undesirable consequences (eg ticket #17330<https://gitlab.haskell.org/ghc/ghc/issues/17330>) * The proposal is to move to Solution D on that page; you can see how it plays out in MR !1970<https://gitlab.haskell.org/ghc/ghc/merge_requests/1970>. * (I think Solutions B and C are non-starters by comparison.) If you care, please check out the design and the MR. We can change later, of course, but doing so changes a lot of code, including client code, so we'd prefer not to. Let's try to converge by the end of the week. Thanks Simon

5 9

Status of the rich errors proposal
by Javier Neira Sanchez 29 Oct '19

29 Oct '19

Hi, lately i am being collaborated in the haskell-ide-engine (hie) repo and one of the issues i get to fix was related with the parsing of ghc errors. It turns out that delimiters of terms in errors are different in windows (`term') and *nix (‘term’) systems and that drive to parse errors. Quite code actions and diagnostics are based in parsing them and they were broken in windows. I am afraid that the code is very brittle and close to human readable error messages. the actual code look like this: -- | Extract a term from a compiler message. -- It looks for terms delimited between '‘' and '’' falling back to '`' and '\'' -- (the used ones in Windows systems). extractTerm :: T.Text -> T.Text extractTerm txt = case extract '‘' '’' txt of "" -> extract '`' '\'' txt -- Needed for windows term -> term where extract b e = T.dropWhile (== b) . T.dropWhileEnd (== e) . T.dropAround (\c -> c /= b && c /= e) or extractImportableTerm :: T.Text -> Maybe (T.Text, SymbolImport SymbolType) extractImportableTerm dirtyMsg = do (n, s) <- extractedTerm let n' = T.strip n return (n', s) where importMsg = S.headMay -- Get rid of the rename suggestion parts $ T.splitOn "Perhaps you meant " $ T.replace "\n" " " -- Get rid of trailing/leading whitespace on each individual line $ T.unlines $ map T.strip $ T.lines $ T.replace "* " "" -- Needed for Windows $ T.replace "• " "" dirtyMsg extractTerm prefix symTy = importMsg >>= T.stripPrefix prefix >>= \name -> Just (name, Import symTy) extractType b = extractTerm ("Not in scope: type constructor or class " <> b) Type extractedTerm = asum [ extractTerm "Variable not in scope: " Symbol , extractType "‘" , extractType "`" -- Needed for windows , extractTerm "Data constructor not in scope: " Constructor] It is clearly unsatisfactory but hard to improve without changing the messages to make it more structured. Moreover any legitimate change on errors to make it better will likely break it. After exposing my worries in the hie irc channel, @mpickering pointed out that it is already a proposal to improve error messages: https://github.com/bgamari/ghc-proposals/blob/rich-errors-proposal/proposal… that nicely will improve the state of things. Otoh there are already a way to output ghc errors as json (see https://gitlab.haskell.org/ghc/ghc/issues/13190) It contains valuable info about the error in specific fields but the message itself is in plain text. So merging both features will let tools to handle compiler errors without use the ghc api directly if needed. what is the status of the proposal? hie an other tooling developers will welcome it very heartly. Thanks in advance!

3 2

Simplifier bug fixed in GHC 8.8.1?
by Alexis King 28 Oct '19

28 Oct '19

Hi all, I have an odd question: I’ve bumped into a clear simplifier bug, and although it only happens on GHC 8.6.5, not 8.8.1, I’d like to locate the change that fixed it. My library’s test suite currently fails on GHC 8.6.5 due to the bug, and I’d rather not force all my users to upgrade to 8.8 if I can help it, so I’m hoping to find a workaround. The minimal test case I’ve found for the bug is this program: {-# LANGUAGE GeneralizedNewtypeDeriving, StandaloneDeriving, TypeFamilies #-} import Control.Exception import Control.Monad.IO.Class import Control.Monad.Trans.Identity import Control.Monad.Trans.Reader class Monad m => MonadFoo m where foo :: m a -> m a instance MonadFoo IO where foo m = onException m (pure ()) instance MonadFoo m => MonadFoo (ReaderT r m) where foo m = ReaderT $ \r -> foo (runReaderT m r) deriving instance MonadFoo m => MonadFoo (IdentityT m) type family F m where F m = IdentityT m newtype FT m a = FT { runFT :: F m a } deriving (Functor, Applicative, Monad, MonadIO, MonadFoo) main :: IO () main = run (foo (liftIO (throwIO (IndexOutOfBounds "bang")))) where run :: ReaderT () (FT (ReaderT () IO)) a -> IO a run = flip runReaderT () . runIdentityT . runFT . flip runReaderT () Using GHC 8.6.5 on macOS 10.14.5, compiling this program with optimizations reliably triggers the -fcatch-bottoms sanitization: $ ghc -O -fcatch-bottoms weird.hs && ./weird [1 of 1] Compiling Main ( weird.hs, weird.o ) Linking weird ... weird: Bottoming expression returned What goes wrong? Somehow the generated core for this program includes the following: lvl_s47B :: SomeException lvl_s47B = $fExceptionArrayException_$ctoException lvl_s483 m_s47r :: () -> State# RealWorld -> (# State# RealWorld, () #) m_s47r = \ _ (eta_B1 :: State# RealWorld) -> raiseIO# lvl_s47B eta_B1 main_s2Ww :: State# RealWorld -> (# State# RealWorld, () #) main_s2Ww = \ (eta_a2wK :: State# RealWorld) -> catch# (case m_s47r `cast` <Co:33> of { }) raiseIO# eta_a2wK This core is completely bogus: it assumes that m_s47r is bottom, but m_s47r is a top-level function! The program still passes -dcore-lint, unfortunately, as it is still well-typed. (Also, in case it helps: -ddump-simplifier-iterations shows that the buggy transformation occurs in the first iteration of the very first simplifier pass.) I’ve been trying to figure out what change might have fixed this so that I can assess if it’s possible to work around, but I haven’t found anything obvious. I’ve been slowly `git bisect`ing to look for the commit that introduced the fix, but many of the commits I’ve tested cause unrelated panics on my machine, which has been exacerbating the problem of the slow recompilation times. I’m a little at wits’ end, but opening a bug report hasn’t felt right, since the bug does appear to already be fixed. Does this issue ring any bells to anyone on this list? Is there a particular patch that landed between GHC 8.6.5 and GHC 8.8.1 that might have fixed this problem? If not, I’ll keep trying with `git bisect`, but I’d appreciate any pointers. Thanks, Alexis

2 2

Debugging specialization
by Alexis King 28 Oct '19

28 Oct '19

Hi all, I am trying to understand why GHC is not specializing an imported INLINABLE instance method, and the information provided by -ddump-spec -Wall-missed-specialisations has not been enough to help me figure it out. Is there some easy/well-trodden way that I could build GHC with some additional instrumentation in place to help me better understand the decisions being made by the specializer, or is -ddump-spec the most granularity available? I’ve noticed that Specialise.hs has handful of pprTrace calls sprinkled about, but they are all commented out. Is the recommended method to just uncomment these calls and rebuild GHC? Thanks, Alexis

3 3

ghc, rts/Linker.c, OpenBSD and WXNEEDED and NOPIE
by Matthias Kilian 27 Oct '19

27 Oct '19

Hi, tl;dr: with shared / dynamic libs, is anything from rts/Linker.c still in use? If so, where and why? I hope this hasn't already been discussed in the past, and I apolotzie if all this is no longer relevant for ghc-8.8 or newer (I'm still at 8.6)... On OpenBSD, I had to explicitely mark ghc as WXNEEDED (i.e. allow it to map pages both as writable and executable), which I would like to get rid of. Likewise, I had to do some dances to explicitely disable PIE. If my memory doesn't fail completely, most (if not all) oft his was because of rts/Linker.c mmapping objects to be loaded at runtime and kind of simulating what dlopen() does for shared objects. Recently, thanks to Greg Steuck (who helped me a lot with this), we updated our ghc port to ghc-8.6 and also enabled shared (or dynamic, if you prefer this term) libraries. So my hope was that the code in rts/Linker.c were no longer used (because dynamic loading as done by ghci or template Haskell) could be done with dlopen(). However, after building ghc-8.6 with all the WXNNEDED stuff removed, and running ghci from this build, I still get an abort because pages are mapped both writable and executable (I've configured my system to abort processes when a W^X violation happens). Here's a backtrace in case it's helpfull at all: #0 _thread_sys___syscall () at -:3 #1 0x00000002010bb179 in _libc_mmap (addr=Variable "addr" is not available.) at /usr/src/lib/libc/sys/mmap.c:47 #2 0x000000021be22374 in mmapForLinker (bytes=Variable "bytes" is not available.) at rts/Linker.c:1029 #3 0x000000021be3e99b in m32_allocator_init () at rts/linker/M32Alloc.c:160 #4 0x00000002fd518be1 in cimc_info () from /usr/local/lib/ghc/bin/../ghci-8.6.4/libHSghci-8.6.4-ghc8.6.4.so #5 0x0000000000000000 in ?? () Any thougts about this? And, since W|X (and nopie) isn't good on any system: is anyone working on getting rid of this for other unixes? Thanks in advance for any help. Ciao, Kili

2 1

How to navigate around the source tree?
by Matthew Pickering 25 Oct '19

25 Oct '19

Hi, The module rework has broken my workflow. Now my tags file is useless for jumping for modules as there are multiple "Utils" and "Types" modules. Invariable I am jumping to the wrong one. What do other people do to avoid this? Can we either revert these changes or give these modules unique names to facilitate that only reliable way of navigating the code base. Perhaps `HsUtils` and `StgUtils` would be appropriate to disambiguate `Hs/Utils` and `StgToCmm/Utils`. Cheers, Matt

8 12

Urgent: git problem
by Simon Peyton Jones 23 Oct '19

23 Oct '19

Aieee! All my GHC repos are failing with this. As a result I can't pull. What should I do? Thanks! Simon git pull error: cannot lock ref 'refs/remotes/origin/wip/rae/remove-tc-dep': 'refs/remotes/origin/wip/rae' exists; cannot create 'refs/remotes/origin/wip/rae/remove-tc-dep' >From gitlab.haskell.org:ghc/ghc ! [new branch] wip/rae/remove-tc-dep -> origin/wip/rae/remove-tc-dep (unable to update local ref) error: cannot lock ref 'refs/remotes/origin/wip/rae/split-up-modules': 'refs/remotes/origin/wip/rae' exists; cannot create 'refs/remotes/origin/wip/rae/split-up-modules' ! [new branch] wip/rae/split-up-modules -> origin/wip/rae/split-up-modules (unable to update local ref) simonpj@MSRC-3645512:~/code/HEAD-2$

6 9

Why is libraries/ghci built with stage 0 compiler instead of stage 1?
by Ömer Sinan Ağacan 18 Oct '19

18 Oct '19

Stage 1 compiler doesn't have interpreter, and doesn't run plugins or TH, so I think GHCi stuff should not be used by stage 1 compiler, but for some reason the "ghci" library (libraries/ghci) is built with stage 0 compiler instead of stage 1. Anyone know what this is? Thanks, Ömer

2 1

atomicModifyMutVar2
by Simon Peyton Jones 17 Oct '19

17 Oct '19

David I'm deeply puzzled atomicModifyMutVar2#. I have read the proposal<https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0149-a…>, and the comments in primops.txt.pp (reproduced below). Question 1 I think the "real" type of atomicModifyMutVar2 is atomicModifyMutVar2# :: MutVar# s a -> (a -> (a,b)) -> State# s -> (# State# s, a, (a, b) #) Nowhere is this explicitly stated, but I believe that the intended semantics of a call case (atomicModifyMutVar2# mv f s) of (# s', x, r #) -> blah Then, suppose the old value of the MutVar was 'old' * The primop builds a thunk t = f old * The new value of the mutable variable is (fst t) * The result r is t * The result x is old Question: is that correct? We should state it explicitly. Question 2 Next question: Why does f have to return a pair? So far as I can tell, it's only so that a client can force it. The 'b' part never seems to play a useful role. So we could equally well have had atomicModifyMutVar2# :: MutVar# s a -> (a -> Box a) -> State# s -> (# State# s, a, Unit a #) where Unit is defined in Data.Tuple data Unit a = Unit a Now you can force the result of (f old), just as with a pair. But the 'b' would no longer complicate matters. Question: is the 'b' in the pair significant? Or could we use Unit? Question 3 In the comments below you say "but we don't know about pairs here". Are you sure? What stops you importing Data.Tuple into GHC.Prim? This fancy footwork is one more complication, if it could be avoided. Thanks Simon primop AtomicModifyMutVar2Op "atomicModifyMutVar2#" GenPrimOp MutVar# s a -> (a -> c) -> State# s -> (# State# s, a, c #) { Modify the contents of a {\tt MutVar\#}, returning the previous contents and the result of applying the given function to the previous contents. Note that this isn't strictly speaking the correct type for this function; it should really be {\tt MutVar\# s a -> (a -> (a,b)) -> State\# s -> (\# State\# s, a, (a, b) \#)}, but we don't know about pairs here. } with out_of_line = True has_side_effects = True can_fail = True

3 12

Re: [Reminder] Merge window for 8.10 closing
by Ryan Scott 16 Oct '19

16 Oct '19

I've been working on a fix for #13154 [1] that I'd like to get into GHC 8.10. My patch [2] is mostly done; I just need to find the time to add comments and perform some slight code cleanup. (Hopefully this week!) Ryan S. ----- [1] https://gitlab.haskell.org/ghc/ghc/issues/13154 [2] https://gitlab.haskell.org/RyanGlScott/ghc/tree/wip/T13154-take-two

1 0