- ghc-devs - Haskell.org

GHCs dependencies (libraries) and maintenance
by Moritz Angermann 01 Jun '20

01 Jun '20

Hi there! so this comes up periodically, and I think we need to discuss this. This is not related to anything right now, so if you wonder if I'm writing this because of something that just happened that I'm involved and you might have missed something, you probably did not. It came up on the #ghc IRC channel a few day ago. GHC depends on quite a set of libraries, and ships those in releases. When ever a new GHC release is cut, all these dependencies need to be on hackage and have release versions. We do not want to produce a GHC release which depends on in-flight packages. In-flight might happen for example due to GHC having to patch dependencies to make them work with HEAD. Everyone who maintains any kind of software online, knows that maintenance can be a drag, and then life happens, and what not. There are many very responsive maintainers and we all owe them a grate amount of gratitude towards their relentless work, keeping those libraries up to date and responding to questions, patches, ... I therefore would like to float the following idea to make the GHC release processes a bit more reliable. GHCHQ (that is those in charge of producing GHC releases for us all), are made co-maintainers on each library GHC depends on, to guarantee that GHC can move forward in the worst of circumstances. Now I would hope that in almost all cases GHCHQ would never have to maintain any of the dependencies actively, they deal with GHC already, so let's try to keep it that way. However GHCHQ can, after a 14day notification period, exercise the co-maintainance and cut releases (and upload them to hackage), should the maintainer not be able to do so on his own for various reasons. I'd like to see this as an insurance policy for GHC continuous development. The only alternative that I see would be that GHCHQ starts forking dependencies and initiates the hackage maintainer takeover protocol, which will cause additional delays, and incurs an extra burden on the GHC maintainers. I hope we can all agree that libraries that end up being dependencies of GHC should be held in high regards and form the very foundation GHC is built upon. As such it should be an honour to have GHCHQ being a co-maintainer for ones library, as it signifies that importances of the library for the continuous development of GHC. Again I don't expect much to change, except for GHCHQ becoming co-maintainers for libraries GHC depends on. The baseline expectation will remain as it is. However we will have ensured the frictionless development of GHC going forward. Cheers, Moritz

4 4

[hadrian] Use LLVM for test and validate target
by Stefan Schulze Frielinghaus 30 May '20

30 May '20

Hi all, Is it possible to force Hadrian to use the LLVM code generator for the test and validate targets? For example, while using the following commands it seems to me that only the native code generator (if available) is used: $ ./hadrian/build -c -j --flavour=quick-llvm test $ ./hadrian/build -c -j --flavour=validate Cheers, Stefan

2 1

STG/CMM questions
by Daneel Yaitskov 29 May '20

29 May '20

Hi, There are 2 questions: 1. How to decipher myThreadId implementation in STG? What is the principle behind STG version? Is it matching first field from TSO by type? "id" is not mentioned anyhow and there is no offset to id field. The whole TSO struct is just mentioned. emitAssign and CmmLocal are used everywhere, so they don't express logic particular to myThreadId function. was: return tso->id; now: currentTSOExpr = CmmReg currentTSOReg MyThreadIdOp -> \[] -> opAllDone $ \[res] -> do emitAssign (CmmLocal res) currentTSOExpr 2. Cmm macro for new TSO field is not found. I added to deriveConstants/Main.hs: ,closureField C "StgTSO" "trace_id" TSO.h: typedef struct StgTSO_ { ... StgClosure* trace_id; PrimsOps.cmm: gcptr p; p = StgTSO_trace_id(CurrentTSO); Linking error happens after make clean && ./boot && ./configure && make -j8 so I guess macros is not defined. Looks like such macros are not written to any file and it is hard to grep them. collect2: error: ld returned 1 exit status `cc' failed in phase `Linker'. (Exit code: 1) utils/iserv/ghc.mk:101: recipe for target 'utils/iserv/stage2/build/tmp/ghc-iserv' failed make[1]: *** [utils/iserv/stage2/build/tmp/ghc-iserv] Error 1 /home/dan/demo/haskell/compiler/ghc/rts/dist/build/libHSrts_thr_p.a(PrimOps.thr_p_o):function stg_getAdamTraceIdzh: error: undefined reference to 'StgTSO_trace_id' /home/dan/demo/haskell/compiler/ghc/rts/dist/build/libHSrts_thr_p.a(PrimOps.thr_p_o):function stg_getAdamTraceIdzh: error: undefined reference to 'n' collect2: error: ld returned 1 exit status `cc' failed in phase `Linker'. (Exit code: 1) utils/iserv/ghc.mk:105: recipe for target 'utils/iserv/stage2_p/build/tmp/ghc-iserv-prof' failed make[1]: *** [utils/iserv/stage2_p/build/tmp/ghc-iserv-prof] Error 1 Makefile:123: recipe for target 'all' failed -- Best regards, Daniil Iaitskov

2 1

Re: keepAlive# primop
by Ben Gamari 27 May '20

27 May '20

Ccing ghc-devs@ since this discussion is something of general interest to the community. Sylvain Henry <sylvain(a)haskus.fr> writes: > Simon, Ben, > > I've been reading and thinking about `readRW#` issues which are very > related to issues we have with `keepAlive#` primop. > > To recap, the problem is that we want some transformations (that Simon > has listed in [1]) to consider: > > ``` > case runRW# f of ... > > case keepAlive# k a of ... > ``` > > as if they were really: > > ``` > case f realWorld# of ... > > case a of ... > ``` > > BUT without breaking the semantics of runRW# and keepAlive#. > > I have been thinking about a solution that I have described on the wiki: > https://gitlab.haskell.org/ghc/ghc/-/wikis/proposal/with-combinator#option-… > > The idea is to keep a set of variable names in each Core case-expression > that are kept alive during the evaluation of the scrutinee. > > I think it would work very nicely with your `newState#` primop described > in [2], both for `runST` and for `unsafeDupablePerformIO` (details on > the wiki). > > It requires a little more upfront work to adapt the code involving > case-expressions. But it will force us to review all transformations to > check if they are sound when keep-alive sets are not empty, which we > would have to do anyway if we implemented another option. We could start > by disabling transformations involving non-empty keep-alive sets and > iterate to enable the sound ones. > > I would like your opinions on the approach. I may have totally missed > something. Thanks for writing this down! Indeed it is an interesting idea. However, as expressed on IRC, I wonder whether this problem rises to the level where it warrants an adaptation to our Core representation. It feels a bit like the tail is wagging the dog here, especially given how the "tail" here merely exists to support FFI. That being said, this is one of the few options which remain on the table that doesn't require changes to user code. Moreover, the applicability to runRW# is quite intriguing. Another (admittedly, more ad-hoc) option that would avoid modifying Core would be to teach the simplifier about the class of "continuation-passing" primops (e.g. `keepAlive#` and `runRW#`), allowing it to push case analyses into the continuation argument. That is, case keepAlive# x expr of pat -> rhs ~> keepAlive# x (case expr of pat -> rhs) Of course, doing this is a bit tricky since one must rewrite the application of keepAlive# to ensure that the resulting application is well-typed. Admittedly, this doesn't help the runRW# case (although this could presumably be accommodated by touch#'ing the final state token in the runRW# desugaring emitted by CorePrep). On the whole, I'm not a fan of this ad-hoc option. It increases the complexity of the simplifier all to support a single operation. By comparison, the Core extension looks somewhat appealing. Cheers, - Ben [1] https://gitlab.haskell.org/ghc/ghc/-/wikis/proposal/with-combinator#option-… P.S. A minor note: the keepAlive# "pseudo-instruction" mentioned on the Wiki [1] is precisely the touch# operation we have today.

3 2

in-tree API Annotations
by Alan & Kim Zimmerman 26 May '20

26 May '20

I am trying to get my head around the best way to put the API annotations into the GHC ParsedSource directly, so ghc-exactprint can move in to GHC and make the development/testing process easier. Prior to API Annotations, there were very few locations in the AST, as they were added as anchor points for the annotations, being separate from the AST and indexed to it by the SrcSpan. I am intending to strip out the superfluous ones, in particular all the (Located RdrName), replacing them with something like data ApiAnnName a = N ApiAnn a instead. So they would still be wrapped, but the annotations would be only ones that can occur for a RdrName, such a backticks, parens, and so on. Does anything actually need these locations now they are there? I am thinking specifically of the .hie files, as the `toHie` instances seem to make use of locations extensively. Regards Alan

1 1

8.12 plans
by Ben Gamari 24 May '20

24 May '20

Hi everyone, The time is again upon us to start thinking about release planning for the next major release: GHC 8.12. In keeping with our 6-month release schedule, I propose the following schedule: * Mid-June 2020: Aim to have all major features in the tree * Late-June 2020: Cut the ghc-8.12 branch * June - August 2020: 3 alpha releases * 1 September 2020: beta release * 25 September 2020: Final 8.12.1 release So, if you have any major features which you would like to merge for 8.12, now is the time to start planning how to wrap them up in the next month or so. As always, do let me know if you think this may be problematic and we can discuss options. Cheers, - Ben

6 7

profiling a constructor?
by Richard Eisenberg 24 May '20

24 May '20

Hi devs, Is there a way to count the number of times a particular constructor is allocated? I might want to know, say, the total number of cons cells allocated over the course of a program, or (in my actual case) the total number of FunTy cells allocated over the course of a program. I can try to do this with an SCC. But it's very clunky: * The constructor may be used in a number of places; each such place would need the SCC. * SCCs can interfere with optimizations. In my use case, this would negate the usefulness of the exercise entirely, as I think some of the structures I wish to observe should never come into being, due to optimizations (e.g. case-of-known-constructor after inlining). Surely, the RTS must be able to count the number of times a constructor is used. But is there any way to access such a feature? If others agree that this would sometimes be helpful, perhaps we can build the feature. Now is not the first time I've wanted this. Thanks! Richard

5 6

How to use the compact regions API to write a compact to a file?
by Matthew Pickering 22 May '20

22 May '20

Dear devs, I have been under the impression for a while that you can write a compact region to a file and the functions in GHC.Compact.Serialized also seem to suggest this is possible. https://hackage.haskell.org/package/ghc-compact-0.1.0.0/docs/GHC-Compact-Se… It seems like there are some functions missing from the API. There is "importCompactByteStrings" but no function in the API which produces ByteStrings. So in order to use this function you have to convert a SerializedCompact into a ByteString, but in what specific way? Has anyone got any code where they have tried to do this? Cheers, Matt

2 2

build ghc with make (ghc-boot-th broken deps)
by Daneel Yaitskov 21 May '20

21 May '20

Hi List, I pulled ghc repo (568d7279a) and trying to build with following env: Ghc from stack (8.8.3). I made links to ghc and ghc-pkg in /usr/bin ./boot is passing well, but ./configure breaks on ghc-boot step. I have no idea how could I fix this issue. I manually modified dep versions in to what I have for ghc-8.8.3 and issue with ghc-boot-th disappeared and appeared parsec libraries/bootstrapping.conf/binary-0.8.7.0.conf depends: array-0.5.4.0 base-4.13.0.0 bytestring-0.10.10.0 containers-0.6.2.1 but there was depends: array-0.5.3.0 base-4.12.0.0 bytestring-0.10.8.2 containers-0.6.0.1 Where is actual problem? Old cabal? 10:45$ cabal --version cabal-install version 1.24.0.2 compiled using version 1.24.2.0 of the Cabal library Configuring ghc-boot-th-8.11.0.20200520... "/usr/bin/ghc-pkg" update -v0 --force --package-db=libraries/bootstrapping.conf libraries/ghc-boot-th/dist-boot/inplace-pkg-config "inplace/bin/ghc-cabal" configure libraries/ghc-boot dist-boot --with-ghc="/usr/bin/ghc" --with-ghc-pkg="/usr/bin/ghc-pkg" --package-db=/home/dan/demo/haskell/compiler/ghc/libraries/bootstrapping.conf --disable-library-for-ghci --enable-library-vanilla --enable-library-for-ghci --disable-library-profiling --disable-shared --with-hscolour="/home/dan/.cabal/bin/HsColour" --configure-option=CFLAGS="-Wall -Werror=unused-but-set-variable -Wno-error=inline -iquote /home/dan/demo/haskell/compiler/ghc/libraries/ghc-boot" --configure-option=LDFLAGS=" " --configure-option=CPPFLAGS=" " --gcc-options="-Wall -Werror=unused-but-set-variable -Wno-error=inline -iquote /home/dan/demo/haskell/compiler/ghc/libraries/ghc-boot " --constraint "binary == 0.8.7.0" --constraint "transformers == 0.5.6.2" --constraint "mtl == 2.2.2" --constraint "hpc == 0.6.1.0" --constraint "ghc-boot-th == 8.11.0.20200520" --constraint "ghc-boot == 8.11.0.20200520" --constraint "template-haskell == 2.17.0.0" --constraint "text == 1.2.4.0" --constraint "parsec == 3.1.14.0" --constraint "Cabal == 3.3.0.0" --constraint "ghc-heap == 8.11.0.20200520" --constraint "exceptions == 0.10.4" --constraint "ghci == 8.11.0.20200520" --constraint "terminfo == 0.4.1.4" --with-gcc="gcc" --with-ld="ld.gold" --with-ar="ar" --with-alex="/home/dan/.cabal/bin/alex" --with-happy="/home/dan/.cabal/bin/happy" Configuring ghc-boot-8.11.0.20200520... Error: The following packages are broken because other packages they depend on are missing. These broken packages must be rebuilt before they can be used. installed package binary-0.8.7.0 is broken due to missing package array-0.5.3.0, base-4.12.0.0, bytestring-0.10.8.2, containers-0.6.0.1 libraries/ghc-boot/ghc.mk:3: recipe for target 'libraries/ghc-boot/dist-boot/package-data.mk' failed make[1]: *** [libraries/ghc-boot/dist-boot/package-data.mk] Error 1 Makefile:123: recipe for target 'all' failed make: *** [all] Error 2 -- Best regards, Daniil Iaitskov

2 3

Brief GitLab reboot
by Ben Gamari 19 May '20

19 May '20

Hi all, GitLab will be down for about 10 minutes for a quick reboot. Apologies for the inconvenience. Cheers, - Ben

1 0