April 2020 - ghc-devs

The future of Phabricator
by Ben Gamari 18 Sep '21

18 Sep '21

TL;DR. For several reasons I think we should consider alternatives to Phabricator. My view is that GitLab seems like the best option. Hello everyone, Over the past year I have been growing increasingly weary of our continued dependence on Phabricator. Without a doubt, its code review interface is the best I have used. However, for a myriad of reasons I am recently questioning whether it is still the best tool for GHC's needs. # The problem There are a number of reasons why I am currently uncertain about Phabricator. For one, at this point we have no options for support in the event that something goes wrong as the company responsible for Phabricator, Phacility, has closed their support channels to non-paying customers. Furthermore, in the past year or two Phacility has been placing their development resources in the parts their customers pay them for, which appear to be much different that the parts that we actively use. For this reason, some parts that we rely on seem oddly half-finished. This concern was recently underscored by some rather unfortunate misfeatures in Harbormaster which resulted in broken CI after the Hadrian merge and now apparent bugs which have made it difficult to migrate to the CircleCI integration we previously prepared. Perhaps most importantly, in our recent development priorities survey our use of Phabricator was the most common complaint by a fair margin, both in case of respondents who have contributed patches and those who have not. On the whole, past contributors and potential future contributors alike have strongly indicated that they want a more Git-like experience. Of course, this wasn't terribly surprising; this is just the most recent case where contributors have made this preference known. Frankly, in a sense it is hard to blame them. The fact that users need to install a PHP tool, Arcanist, to contribute anything but documentation patches has always seemed like unnecessary friction to me and I would be quite happy to be rid of it. Indeed we have had a quite healthy number of GitHub documentation patches since we started accepting them. This makes me thing that there may indeed be potential contributoes that we are leaving on the table. # What to do With Rackspace support ending at the end of year, now may be a good time to consider whether we really want to continue on this road. Phabricator is great at code review but I am less and less certain that it is worth the maintenance uncertainty and potential lost contributors that it costs. Moreover, good alternatives seem closer at-hand than they were when we deployed Phabricator. ## Move to GitHub When people complain about our infrastructure, they often use GitHub as the example of what they would like to see. However, realistically I have a hard time seeing GitHub as a viable option. Its feature set is simply insufficient enough to handle the needs of a larger project like GHC without significant external tooling (as seen in the case of Rust-lang). The concrete reasons have been well-documented in previous discussions but, to summarize, * its review functionality is extremely painful to use with larger patches * rebased patches lead to extreme confusion and lost review comments * it lacks support for pre-receive hooks, which serve as a last line of defense to prevent unintended submodule changes * its inability to deal with external tickets is problematic * there is essentially no possibility that we could eventually migrate GHC's tickets to GitHub's ticket tracker without considerable data loss (much less manage the ticket load that would result), meaning that we would forever be stuck with maintaining Trac. * on a personal note, its search functionality has often left me empty-handed On the whole, these issues seem pretty hard to surmount. ## Move to GitLab In using GitLab for another project over the past months I have been positively surprised by its quality. It handles rebased merge requests far better than GitHub, has functional search, and quite a usable review interface. Furthermore, upstream has been extremely responsive to suggestions for improvement [1]. Even out-of-the-box it seems to be flexible enough to accommodate our needs, supporting integration with external issue trackers, having reasonable release management features, and support for code owners to automate review triaging (replacing much of the functionality of Phabricator's Herald). Finally, other FOSS projects' [3] recent migrations from Phabrictor to GitLab have shown that GitLab-the-company is quite willing to offer help when needed. I took some time this weekend to try setting up a quick GHC instance [2] to play around with. Even after just a few hours of playing around I think the result is surprisingly usable. Out of curiosity I also played around with importing some tickets from Trac (building on Matt Pickering's Trac-to-Maniphest migration tool). With relatively little effort I was even able to get nearly all of our tickets (as of 9 months ago) imported while preserving ticket numbers (although there are naturally a few wrinkles that would need to be worked out). Naturally, I think we should treat the question of ticket tracker migration as an orthogonal one to code review, but it is good to know that this is possible. ## Continue with Phabricator Continuing with Phabricator is of course an option. Its review functionality is great and it has served us reasonably well. However, compared to GitLab and even GitHub of today its features seem less distinguished than they once did. Moreover, the prospect of having to maintain a largely stagnant product with no support strikes me as a slightly dangerous game to play. Working around the issues we have recently encountered has already cost a non-negligible amount of time. # The bottom line If it wasn't clear already, I think that we should strongly consider a move to GitLab. At this point it seems clear that it isn't going to vanish, has a steady pace of development, is featureful, and available. However, these are just my thoughts. What do you think? Cheers, - Ben [1] 11.4 will ship with a file tree view in the code review interface, which I reported (https://gitlab.com/gitlab-org/gitlab-ce/issues/46474) as being is one of the Phabricator features I missed the most during review [2] https://gitlab.home.smart-cactus.org/ghc/ghc/issues/14641 [3] The GNOME and freedesktop.org projects have recently migrated, the former from a hodge-podge of self-hosted services and the latter from Phabricator

17 40

New implementation for `ImpredicativeTypes`
by Alejandro Serrano Mena 08 Sep '21

08 Sep '21

Hi all, As I mentioned some time ago, we have been busy working on a new implementation of `ImpredicativeTypes` for GHC. I am very thankful to everybody who back then sent us examples of impredicativity which would be nice to support, as far as we know this branch supports all of them! :) If you want to try it, at https://gitlab.haskell.org/trupill/ghc/commit/a3f95a0fe0f647702fd7225fa719a… you can find the result of the pipeline, which includes builds for several platforms (click on the "Artifacts" button, the one which looks like a cloud, to get them). The code is being developed at https://gitlab.haskell.org/trupill/ghc. Any code should run *unchanged* except for some eta-expansion required for some specific usage patterns of higher-rank types. Please don't hesitate to ask any questions or clarifications about it. A merge request for tracking this can be found at https://gitlab.haskell.org/ghc/ghc/merge_requests/1659 Kind regards, Alejandro

8 23

Re: keepAlive# primop
by Ben Gamari 27 May '20

27 May '20

Ccing ghc-devs@ since this discussion is something of general interest to the community. Sylvain Henry <sylvain(a)haskus.fr> writes: > Simon, Ben, > > I've been reading and thinking about `readRW#` issues which are very > related to issues we have with `keepAlive#` primop. > > To recap, the problem is that we want some transformations (that Simon > has listed in [1]) to consider: > > ``` > case runRW# f of ... > > case keepAlive# k a of ... > ``` > > as if they were really: > > ``` > case f realWorld# of ... > > case a of ... > ``` > > BUT without breaking the semantics of runRW# and keepAlive#. > > I have been thinking about a solution that I have described on the wiki: > https://gitlab.haskell.org/ghc/ghc/-/wikis/proposal/with-combinator#option-… > > The idea is to keep a set of variable names in each Core case-expression > that are kept alive during the evaluation of the scrutinee. > > I think it would work very nicely with your `newState#` primop described > in [2], both for `runST` and for `unsafeDupablePerformIO` (details on > the wiki). > > It requires a little more upfront work to adapt the code involving > case-expressions. But it will force us to review all transformations to > check if they are sound when keep-alive sets are not empty, which we > would have to do anyway if we implemented another option. We could start > by disabling transformations involving non-empty keep-alive sets and > iterate to enable the sound ones. > > I would like your opinions on the approach. I may have totally missed > something. Thanks for writing this down! Indeed it is an interesting idea. However, as expressed on IRC, I wonder whether this problem rises to the level where it warrants an adaptation to our Core representation. It feels a bit like the tail is wagging the dog here, especially given how the "tail" here merely exists to support FFI. That being said, this is one of the few options which remain on the table that doesn't require changes to user code. Moreover, the applicability to runRW# is quite intriguing. Another (admittedly, more ad-hoc) option that would avoid modifying Core would be to teach the simplifier about the class of "continuation-passing" primops (e.g. `keepAlive#` and `runRW#`), allowing it to push case analyses into the continuation argument. That is, case keepAlive# x expr of pat -> rhs ~> keepAlive# x (case expr of pat -> rhs) Of course, doing this is a bit tricky since one must rewrite the application of keepAlive# to ensure that the resulting application is well-typed. Admittedly, this doesn't help the runRW# case (although this could presumably be accommodated by touch#'ing the final state token in the runRW# desugaring emitted by CorePrep). On the whole, I'm not a fan of this ad-hoc option. It increases the complexity of the simplifier all to support a single operation. By comparison, the Core extension looks somewhat appealing. Cheers, - Ben [1] https://gitlab.haskell.org/ghc/ghc/-/wikis/proposal/with-combinator#option-… P.S. A minor note: the keepAlive# "pseudo-instruction" mentioned on the Wiki [1] is precisely the touch# operation we have today.

3 2

[ANNOUNCE] Glasgow Haskell Compiler 8.10.1 released
by Ben Gamari 18 May '20

18 May '20

Hello all, The GHC team is happy to announce the availability of GHC 8.10.1. Source and binary distributions are available at the usual place: https://downloads.haskell.org/ghc/8.10.1/ GHC 8.10.1 brings a number of new features including: * The new UnliftedNewtypes extension allowing newtypes around unlifted types. * The new StandaloneKindSignatures extension allows users to give top-level kind signatures to type, type family, and class declarations. * A new warning, -Wderiving-defaults, to draw attention to ambiguous deriving clauses * A number of improvements in code generation, including changes * A new GHCi command, :instances, for listing the class instances available for a type. * An upgraded Windows toolchain lifting the MAX_PATH limitation * A new, low-latency garbage collector. * Improved support profiling, including support for sending profiler samples to the eventlog, allowing correlation between the profile and other program events Note that at the moment we still require that macOS Catalina users exempt the binary distribution from the notarization requirement by running `xattr -cr .` on the unpacked tree before running `make install`. This situation will hopefully be improved for GHC 8.10.2 with the resolution of #17418 [1]. Cheers, - Ben [1] https://gitlab.haskell.org/ghc/ghc/issues/17418

2 3

[HiW'20] Call for Talks
by Ben Gamari 14 May '20

14 May '20

Hello everyone, Haskell Implementors Workshop is calling for talk proposals. Co-located with ICFP, HiW is an ideal place to describe a Haskell library, a Haskell extension, compiler, works-in-progress, demo a new Haskell-related tool, or even propose future lines of Haskell development. The deadline for submissions is July 2nd 2020. Call for Talks ============== The 12th Haskell Implementors’ Workshop is to be held alongside ICFP 2020 this year in New Jersey. It is a forum for people involved in the design and development of Haskell implementations, tools, libraries, and supporting infrastructure, to share their work and discuss future directions and collaborations with others. Talks and/or demos are proposed by submitting an abstract, and selected by a small program committee. There will be no published proceedings. The workshop will be informal and interactive, with open spaces in the timetable and room for ad-hoc discussion, demos and lightning talks. Scope and Target Audience ------------------------- It is important to distinguish the Haskell Implementors’ Workshop from the Haskell Symposium which is also co-located with ICFP 2020. The Haskell Symposium is for the publication of Haskell-related research. In contrast, the Haskell Implementors’ Workshop will have no proceedings – although we will aim to make talk videos, slides and presented data available with the consent of the speakers. The Implementors’ Workshop is an ideal place to describe a Haskell extension, describe works-in-progress, demo a new Haskell-related tool, or even propose future lines of Haskell development. Members of the wider Haskell community encouraged to attend the workshop – we need your feedback to keep the Haskell ecosystem thriving. Students working with Haskell are specially encouraged to share their work. The scope covers any of the following topics. There may be some topics that people feel we’ve missed, so by all means submit a proposal even if it doesn’t fit exactly into one of these buckets: - Compilation techniques - Language features and extensions - Type system implementation - Concurrency and parallelism: language design and implementation - Performance, optimization and benchmarking - Virtual machines and run-time systems - Libraries and tools for development or deployment Talks ----- We invite proposals from potential speakers for talks and demonstrations. We are aiming for 20-minute talks with 5 minutes for questions and changeovers. We want to hear from people writing compilers, tools, or libraries, people with cool ideas for directions in which we should take the platform, proposals for new features to be implemented, and half-baked crazy ideas. Please submit a talk title and abstract of no more than 300 words. Submissions can be made via HotCRP at https://icfp-hiw20.hotcrp.com/ until July 2nd (anywhere on earth). We will also have lightning talks session. These have been very well received in recent years, and we aim to increase the time available to them. Lightning talks be ~7mins and are scheduled on the day of the workshop. Suggested topics for lightning talks are to present a single idea, a work-in-progress project, a problem to intrigue and perplex Haskell implementors, or simply to ask for feedback and collaborators. Logistics --------- We recognize the on-going threat that COVID-19 poses to our participants' safety. While August is nearly half a year away, we must account for the possibility that the virus continues to pose a significant threat into the summer. For this reason, we are investigating options that would allow remote presentations this year. In light of this, we urge potential presenters not to be discouraged from submitting and encourage participants to keep time open in their calendars, regardless of the on-going COVID situation. Rest assured that the conference organizers are working to ensure that the Implementors Workshop can be held safely and productively, regardless of how the COVID-19 situation evolves. Program Committee ----------------- - Andrey Mokhov (Newcastle University) - Ben Gamari (Well-Typed LLP) - Christian Baaij (QBayLogic) - George Karachalias (Tweag I/O) - Klara Marntirosian (KU Leuven) - Matthew Pickering (Univeristy of Bristol) - Ryan Scott (Indiana University Bloomington) Best wishes, Ben

1 1

Connecting the Wolfram Language to Haskell
by Simon Peyton Jones 30 Apr '20

30 Apr '20

Friends Short summary: can someone familiar with using the GHC API offer advice on getting the Wolfram Language connected to Haskell? Many of you will know of the Wolfram Language<https://www.wolfram.com/language/>, which is IMHO pretty cool and not well enough known. There are lots of interesting things about it, but a huge thing is that it's deeply connected to the vast pile of carefully curated data that Wolfram Research has gathered and organised over several decades. Anyway, they'd like to be able to invoke Haskell code from the Wolfram Language interpreter. But as you'll see below, they are understandably stumbling on the engineering aspects of doing so. Rather than spam Stephen Wolfram himself, I'm cc'ing Todd Gayley who is the team lead. (Hi Todd.) I'd really appreciate it if some of you could help advise him. It would be terrific to have a direct connection between Wolfram and Haskell! Many thanks Simon From: Stephen Wolfram Sent: 01 April 2020 01:49 To: Simon Peyton Jones <simonpj(a)microsoft.com> Subject: Haskell question, etc. Simon --- [...snip...] What brings me to email is a Haskell question: It's probably best that I just forward what my team wants to ask: ========= A powerful feature of the Wolfram Language is that it allows users to evaluate code in other programing languages, returning the results to a Wolfram notebook (https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Freference…) We have implemented this feature for a number of languages, including Python, NodeJS, Julia, and Ruby. We would very much like to include Haskell in this set, but we have little Haskell expertise, and have hit a stumbling block. We would appreciate it if some Haskell experts could help us out with this feature. We have the following basic line of code for evaluating a string of Haskell code: r <- liftIO $ runInterpreter $ do { setImports ["Prelude"]; eval strToEvaluate} The problem is that this is a one-shot evaluation, and we want a long-lived interactive Haskell session, to which a series of inputs can be directed. We have been told that we have to use GHCi for that, but we don't know how to do it. The basic flow of our functionality is as follows: 1) User calls StartExternalSession["LanguageName"] to launch an interpreter for the language. This process remains running and can be used for multiple calls. 2) User calls ExternalEvaluate[session, "some code"] to execute the given code in the external language and return a result converted into native Wolfram Language types (strings, numbers, lists, associations, etc.) ZeroMQ is used as the transport between Wolfram Language and the external language interpreter, and JSON is the data format we use to return results. Beyond the basics of hooking up GHCi for this type of session use, we also would like assistance in introspection of Haskell results so that they can be sent back to WL in the most useful form. This is the general structure of what we do for other languages: if result is an object return object data else if result is a function return function data else return JSON form of the expression We have attached a simple file of Haskell code that one of our engineers has successfully used to get a basic evaluation of Haskell code from the Wolfram Language, but it uses the single-shot evaluation code that was given above, and so is not suitable. We would appreciate any help that you can give us, or developers or resources you can point us at, to assist in integrating Haskell into our ExternalEvaluate system. ======== Any help greatly appreciated! (Not least because I'd personally like to play with this... :) ) --- Stephen

2 1

Why doesn’t the simple optimizer fuse casts?
by Alexis King 28 Apr '20

28 Apr '20

This question is spurred by curiosity more than anything else, but I’ve noticed that the simple optimizer doesn’t fuse nested casts, and I’m wondering if there’s any reason it couldn’t. To make what I’m talking about more concrete, suppose we have an expression like this: (x |> co) |> sym co It seems like it would be trivial for simpleOptExpr to fuse the nested casts to get x |> co; sym co and then the coercion optimizer could get rid of it entirely. Moreover, this seems within the spirit of the simple optimizer, since it’s really just “cleaning up” an expression. Is there any reason the simple optimizer doesn’t do this, or is it just something nobody implemented? (For context, I’ve recently been staring at a lot of -ddump-ds output, and there happen to be a bunch of nested casts in the result that are really just noise. It would be nice if the simple optimizer got rid of them for me.) Alexis

4 5

RE: T13456
by Simon Peyton Jones 28 Apr '20

28 Apr '20

Ben I'm still getting framework failures from the testsuite, as below. But now it's not just me: it's CI! See !2600 which is failing in this way. It'd be good to nail this... it seems wrong to have to ignore framework failures when checking that a build validates. Simon | -----Original Message----- | From: Simon Peyton Jones | Sent: 20 April 2020 21:42 | To: Ben Gamari <ben(a)well-typed.com> | Subject: RE: T13456 | | Thanks! | | | -----Original Message----- | | From: Ben Gamari <ben(a)well-typed.com> | | Sent: 20 April 2020 18:57 | | To: Simon Peyton Jones <simonpj(a)microsoft.com>; ghc-devs <ghc- | | devs(a)haskell.org> | | Subject: Re: T13456 | | | | Simon Peyton Jones via ghc-devs <ghc-devs(a)haskell.org> writes: | | | | > I'm getting this failure (below) from validate fairly consistently. | | > It is often silenced by adding an empty file | | > ghci/should_run/T13456.stderr But it's troubling. Does anyone else | see | | this? How can I debug it? | | > | | Indeed this is odd. I have not seen this in CI or my local builds. It's | | possible that I have seen it in local builds that were failing for | other | | reasons but ignored it. | | | | While I don't know why you are seeing these failures in general, the | fact | | that they are reported as framework failures is arguably a bug. I would | | argue that we should treat a non-existing .stderr file as we would an | | empty file. I've opened !3121 fixing this. Hopefully you will see a | more | | helpful error message with this patch. | | | | Cheers, | | | | - Ben

2 2

Desugaring matches with already-desugared RHSs?
by Alexis King 27 Apr '20

27 Apr '20

Hi all, I’m currently in the process of rewriting most of the arrow desugaring code. One of the most awkward parts of the current implementation is the way case commands are desugared. Given a case command like case e1 of A a b -> cmd1 B c -> cmd2 C d e f -> cmd3 the desugarer actually replaces each command on the RHS with an Either-wrapped tuple to get something like this: arr (\env -> case e1 of A a b -> Left (Left (a, b)) B c -> Left (Right c) C d e f -> Right (d, e, f)) >>> ((cmd1 ||| cmd2) ||| cmd3) This means the RHSs of the case expression are really already desugared, and ideally they would be CoreExprs, but matchWrapper expects the RHSs to be HsExprs. The current implementation accommodates this restriction by building fake HsExprs with no location information, but this means the logic for building the tuples in the RHSs has to be duplicated (since other places do want CoreExprs). I was thinking it would be nice to avoid this hack, but I’m not sure what the best way to do it is. One way would be to create a variant of matchWrapper with a type like matchWrapper' :: HsMatchContext GhcRn -> Maybe (LHsExpr GhcTc) -> MatchGroup GhcTc rhs -> (rhs -> DsM CoreExpr) -- how to desugar the RHSs -> DsM ([Id], CoreExpr) and update dsGRHSs to accept an extra argument as well. Then the arrow desugaring code could just pass `return` to matchWrapper' so it wouldn’t touch its RHSs. But I’m not sure if this approach makes sense — nothing else in the desugarer seems to work this way. Is there a better approach I’m not seeing? Thanks, Alexis

2 1

Spam projects on gitlab
by Sylvain Henry 26 Apr '20

26 Apr '20

Hi, I've just noticed these spam projects on our gitlab: - https://gitlab.haskell.org/craigonaldson/lumaslim - https://gitlab.haskell.org/AnthonyMussen/survey-of-evianne-cream - https://gitlab.haskell.org/RobertoHeard/robertoheard - https://gitlab.haskell.org/salihagenter/commission-based-business-in-india Can someone remove them? Thanks, Sylvain

2 1