June 2020 - ghc-devs - Haskell.org

The future of Phabricator
by Ben Gamari 18 Sep '21

18 Sep '21

TL;DR. For several reasons I think we should consider alternatives to Phabricator. My view is that GitLab seems like the best option. Hello everyone, Over the past year I have been growing increasingly weary of our continued dependence on Phabricator. Without a doubt, its code review interface is the best I have used. However, for a myriad of reasons I am recently questioning whether it is still the best tool for GHC's needs. # The problem There are a number of reasons why I am currently uncertain about Phabricator. For one, at this point we have no options for support in the event that something goes wrong as the company responsible for Phabricator, Phacility, has closed their support channels to non-paying customers. Furthermore, in the past year or two Phacility has been placing their development resources in the parts their customers pay them for, which appear to be much different that the parts that we actively use. For this reason, some parts that we rely on seem oddly half-finished. This concern was recently underscored by some rather unfortunate misfeatures in Harbormaster which resulted in broken CI after the Hadrian merge and now apparent bugs which have made it difficult to migrate to the CircleCI integration we previously prepared. Perhaps most importantly, in our recent development priorities survey our use of Phabricator was the most common complaint by a fair margin, both in case of respondents who have contributed patches and those who have not. On the whole, past contributors and potential future contributors alike have strongly indicated that they want a more Git-like experience. Of course, this wasn't terribly surprising; this is just the most recent case where contributors have made this preference known. Frankly, in a sense it is hard to blame them. The fact that users need to install a PHP tool, Arcanist, to contribute anything but documentation patches has always seemed like unnecessary friction to me and I would be quite happy to be rid of it. Indeed we have had a quite healthy number of GitHub documentation patches since we started accepting them. This makes me thing that there may indeed be potential contributoes that we are leaving on the table. # What to do With Rackspace support ending at the end of year, now may be a good time to consider whether we really want to continue on this road. Phabricator is great at code review but I am less and less certain that it is worth the maintenance uncertainty and potential lost contributors that it costs. Moreover, good alternatives seem closer at-hand than they were when we deployed Phabricator. ## Move to GitHub When people complain about our infrastructure, they often use GitHub as the example of what they would like to see. However, realistically I have a hard time seeing GitHub as a viable option. Its feature set is simply insufficient enough to handle the needs of a larger project like GHC without significant external tooling (as seen in the case of Rust-lang). The concrete reasons have been well-documented in previous discussions but, to summarize, * its review functionality is extremely painful to use with larger patches * rebased patches lead to extreme confusion and lost review comments * it lacks support for pre-receive hooks, which serve as a last line of defense to prevent unintended submodule changes * its inability to deal with external tickets is problematic * there is essentially no possibility that we could eventually migrate GHC's tickets to GitHub's ticket tracker without considerable data loss (much less manage the ticket load that would result), meaning that we would forever be stuck with maintaining Trac. * on a personal note, its search functionality has often left me empty-handed On the whole, these issues seem pretty hard to surmount. ## Move to GitLab In using GitLab for another project over the past months I have been positively surprised by its quality. It handles rebased merge requests far better than GitHub, has functional search, and quite a usable review interface. Furthermore, upstream has been extremely responsive to suggestions for improvement [1]. Even out-of-the-box it seems to be flexible enough to accommodate our needs, supporting integration with external issue trackers, having reasonable release management features, and support for code owners to automate review triaging (replacing much of the functionality of Phabricator's Herald). Finally, other FOSS projects' [3] recent migrations from Phabrictor to GitLab have shown that GitLab-the-company is quite willing to offer help when needed. I took some time this weekend to try setting up a quick GHC instance [2] to play around with. Even after just a few hours of playing around I think the result is surprisingly usable. Out of curiosity I also played around with importing some tickets from Trac (building on Matt Pickering's Trac-to-Maniphest migration tool). With relatively little effort I was even able to get nearly all of our tickets (as of 9 months ago) imported while preserving ticket numbers (although there are naturally a few wrinkles that would need to be worked out). Naturally, I think we should treat the question of ticket tracker migration as an orthogonal one to code review, but it is good to know that this is possible. ## Continue with Phabricator Continuing with Phabricator is of course an option. Its review functionality is great and it has served us reasonably well. However, compared to GitLab and even GitHub of today its features seem less distinguished than they once did. Moreover, the prospect of having to maintain a largely stagnant product with no support strikes me as a slightly dangerous game to play. Working around the issues we have recently encountered has already cost a non-negligible amount of time. # The bottom line If it wasn't clear already, I think that we should strongly consider a move to GitLab. At this point it seems clear that it isn't going to vanish, has a steady pace of development, is featureful, and available. However, these are just my thoughts. What do you think? Cheers, - Ben [1] 11.4 will ship with a file tree view in the code review interface, which I reported (https://gitlab.com/gitlab-org/gitlab-ce/issues/46474) as being is one of the Phabricator features I missed the most during review [2] https://gitlab.home.smart-cactus.org/ghc/ghc/issues/14641 [3] The GNOME and freedesktop.org projects have recently migrated, the former from a hodge-podge of self-hosted services and the latter from Phabricator

17 40

New implementation for `ImpredicativeTypes`
by Alejandro Serrano Mena 08 Sep '21

08 Sep '21

Hi all, As I mentioned some time ago, we have been busy working on a new implementation of `ImpredicativeTypes` for GHC. I am very thankful to everybody who back then sent us examples of impredicativity which would be nice to support, as far as we know this branch supports all of them! :) If you want to try it, at https://gitlab.haskell.org/trupill/ghc/commit/a3f95a0fe0f647702fd7225fa719a… you can find the result of the pipeline, which includes builds for several platforms (click on the "Artifacts" button, the one which looks like a cloud, to get them). The code is being developed at https://gitlab.haskell.org/trupill/ghc. Any code should run *unchanged* except for some eta-expansion required for some specific usage patterns of higher-rank types. Please don't hesitate to ask any questions or clarifications about it. A merge request for tracking this can be found at https://gitlab.haskell.org/ghc/ghc/merge_requests/1659 Kind regards, Alejandro

8 23

Perf notes
by Simon Peyton Jones 02 Jul '20

02 Jul '20

Despite a recent git fetch https://gitlab.haskell.org/ghc/ghc-performance-notes.git refs/notes/perf:refs/notes/ci/perf I'm getting lots of perf regressions in HEAD. For example =====> T9203(normal) 1 of 1 [0, 0, 0] ]0;T9203(normal) 1 of 1 [0, 0, 0]cd "T9203.run" && "/home/simonpj/code/HEAD-3/inplace/bin/ghc-stage2" -o T9203 T9203.hs -dcore-lint -dstg-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output -O2< cd "T9203.run" && ./T9203 +RTS -V0 -tT9203.stats --machine-readable -RTS < runtime/bytes allocated increased from x86_64-linux-deb9 baseline @ HEAD~28: Expected T9203 (normal) runtime/bytes allocated: 56046952.0 +/-5% Lower bound T9203 (normal) runtime/bytes allocated: 53244604 Upper bound T9203 (normal) runtime/bytes allocated: 58849300 Actual T9203 (normal) runtime/bytes allocated: 108464536 Deviation T9203 (normal) runtime/bytes allocated: 93.5 % *** unexpected stat test failure for T9203(normal) Performance Metrics (test environment: local): T9203(normal) runtime/bytes allocated 108464536.000 (baseline @ HEAD~28) 56046952.000 [increased, 93.5%] What am I doing wrong? Simon

2 6

perf regression with TypeFamilies
by Richard Eisenberg 02 Jul '20

02 Jul '20

Hi Harendra, I saw your comment on a ghc proposal (https://github.com/ghc-proposals/ghc-proposals/pull/343#issuecomment-650797…) that said you experienced perf regressions with -XTypeFamilies enabled (but no other changes). Are these reproducible in a way that can be shared publicly? And are the regressions in compile times or run times? -XTypeFamilies enables -XMonoLocalBinds, which disables let-generalization on some nested lets. It is thus just barely conceivable that different Core is produced depending on this extension, and that there may be a possibility of performance changes. But this would be unexpected, and something worth investigating. In other words: if you can, do please post a bug -- simply enabling -XTypeFamilies should not slow anything down! Thanks, Richard

2 2

IP/key change for gitlab.haskell.org?
by Richard Eisenberg 26 Jun '20

26 Jun '20

An innocent `git push` yielded this today: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: POSSIBLE DNS SPOOFING DETECTED! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ The ED25519 host key for gitlab.haskell.org has changed, and the key for the corresponding IP address 139.178.85.33 is unknown. This could either mean that DNS SPOOFING is happening or the IP address for the host and its host key have changed at the same time. @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that a host key has just been changed. The fingerprint for the ED25519 key sent by the remote host is SHA256:/dI7zsBRZNPB+0TqskF7rSaZ/LhQw0cF4c5W+4uMlRo. Please contact your system administrator. Add correct host key in /Users/rae/.ssh/known_hosts to get rid of this message. Offending ED25519 key in /Users/rae/.ssh/known_hosts:21 ED25519 host key for gitlab.haskell.org has changed and you have requested strict checking. Host key verification failed. I know the server had a rough weekend. Is this a natural consequence, or is something fishy going on? Thanks, Richard

4 3

[HIW'20] Third Call for Talks
by Ben Gamari 24 Jun '20

24 Jun '20

Hello everyone, Haskell Implementors Workshop is calling for talk proposals. Co-located with ICFP, HiW is an ideal place to describe a Haskell library, a Haskell extension, compiler, works-in-progress, demo a new Haskell-related tool, or even propose future lines of Haskell development. The deadline for submissions is next week, July 2nd 2020. Call for Talks ============== The 12th Haskell Implementors’ Workshop is to be held alongside ICFP 2020 this year. It is a forum for people involved in the design and development of Haskell implementations, tools, libraries, and supporting infrastructure, to share their work and discuss future directions and collaborations with others. Talks and/or demos are proposed by submitting an abstract, and selected by a small program committee. There will be no published proceedings. The workshop will be informal and interactive, with open spaces in the timetable and room for ad-hoc discussion, demos and lightning talks. Scope and Target Audience ------------------------- It is important to distinguish the Haskell Implementors’ Workshop from the Haskell Symposium which is also co-located with ICFP 2020. The Haskell Symposium is for the publication of Haskell-related research. In contrast, the Haskell Implementors’ Workshop will have no proceedings – although we will aim to make talk videos, slides and presented data available with the consent of the speakers. The Implementors’ Workshop is an ideal place to describe a Haskell extension, describe works-in-progress, demo a new Haskell-related tool, or even propose future lines of Haskell development. Members of the wider Haskell community encouraged to attend the workshop – we need your feedback to keep the Haskell ecosystem thriving. Students working with Haskell are specially encouraged to share their work. The scope covers any of the following topics. There may be some topics that people feel we’ve missed, so by all means submit a proposal even if it doesn’t fit exactly into one of these buckets: - Compilation techniques - Language features and extensions - Type system implementation - Concurrency and parallelism: language design and implementation - Performance, optimization and benchmarking - Virtual machines and run-time systems - Libraries and tools for development or deployment Talks ----- We invite proposals from potential speakers for talks and demonstrations. We are aiming for 20-minute talks with 5 minutes for questions and changeovers. We want to hear from people writing compilers, tools, or libraries, people with cool ideas for directions in which we should take the platform, proposals for new features to be implemented, and half-baked crazy ideas. Please submit a talk title and abstract of no more than 300 words. Submissions can be made via HotCRP at https://icfp-hiw20.hotcrp.com/ until July 2nd (anywhere on earth). We will also have lightning talks session. These have been very well received in recent years, and we aim to increase the time available to them. Lightning talks be ~7mins and are scheduled on the day of the workshop. Suggested topics for lightning talks are to present a single idea, a work-in-progress project, a problem to intrigue and perplex Haskell implementors, or simply to ask for feedback and collaborators. Logistics --------- Due to the on-going COVID-19 situation, ICFP (and, consequently, HIW) will be held remotely this year. However, the organizers are still working hard to provide for a great workshop experience. While we are sad that this year will lack the robust hallway track that is often the highlight of HIW, we believe that this remote workshop presents a unique opportunity to include more of the Haskell community in our discussion and explore new modes of communicating with our colleagues. We hope that you will join us in making this HIW as vibrant as any other. Program Committee ----------------- - Andrey Mokhov (Newcastle University) - Ben Gamari (Well-Typed LLP) - Christian Baaij (QBayLogic) - George Karachalias (Tweag I/O) - Klara Marntirosian (KU Leuven) - Matthew Pickering (Univeristy of Bristol) - Ryan G.L. Scott (Indiana University Bloomington) Best wishes, ~ Ben

1 0

Fixing type synonyms to Uniq(D)FM newtypes
by Andreas Klebinger 23 Jun '20

23 Jun '20

There was a discussion about making UniqFM typed for the keys here a while ago. (https://mail.haskell.org/pipermail/ghc-devs/2020-January/018451.html and following) I wrote up an MR for one possible approach here: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/3577 It implements solution 2 from that discussion. Just while getting the patch to typecheck I've already seen a number of cases where this increased readability of the code quite a bit so I think it's a good improvement. If there are strong objections to this solution let me know. In that case I'm happy to abandon the patch. If not I will clean it up and get it ready for merging.

2 2

Re: GitLab status
by Ryan Scott 22 Jun '20

22 Jun '20

Just as an FYI, there appears to be some data loss from this migration. For example, I pushed a commit to !3536 [1] that appears to have been dropped, leading to confusion here [2]. To make things worse, I never received an e-mail notification for this comment—I only discovered it by chance. I also opened an issue about FUN, but that appears to have been lost. I could try to resubmit it, but perhaps the data still exists somewhere on servers somewhere? Ryan S. ----- [1] https://gitlab.haskell.org/ghc/ghc/-/merge_requests/3536 [2] https://gitlab.haskell.org/ghc/ghc/-/merge_requests/3536#note_283322

2 1

Introducing GHC whole program compiler (GHC-WPC)
by Csaba Hruska 22 Jun '20

22 Jun '20

Hello, I've created a whole program compilation pipeline for GHC via STG. Please read my blog post for the details: Introducing GHC whole program compiler (GHC-WPC) <https://www.patreon.com/posts/introducing-ghc-38173710> Regards, Csaba Hruska

3 5

RE: GHC/ | GHC doesn't inline small type class method marked as INLINE with profiling enabled (#18372)
by Simon Peyton Jones 22 Jun '20

22 Jun '20

I get a 404 for https://gitlab.haskell.org/ghc/ghc/-/issues/18372 You may need to re-submit this… s From: Andrzej Rybczak <gitlab(a)gitlab.haskell.org> Sent: 21 June 2020 15:44 To: Simon Peyton Jones <simonpj(a)microsoft.com> Subject: GHC/ | GHC doesn't inline small type class method marked as INLINE with profiling enabled (#18372) Andrzej Rybczak<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.ha…> created an issue: Summary When profiling is enabled, GHC doesn't inline small type class methods which prevents further optimizations and specialization from happening. Steps to reproduce When the following module: module Main where import Optics.Core data T = T { _t1 :: Int , _t2 :: Int } t1 :: Lens' T Int t1 = lensVL $ \f s -> (\n -> s { _t1 = n}) <$> f (_t1 s) {-# INLINE t1 #-} t2 :: Lens' T Int t2 = lensVL $ \f s -> (\n -> s { _t2 = n}) <$> f (_t2 s) {-# INLINE t2 #-} t_val :: T t_val = T 1 2 {-# NOINLINE t_val #-} main :: IO () main = putStrLn . show $ view t1 t_val + view t2 t_val is compiled with profiling enabled, here's how core of main looks: main1 = case ((($fStrongForget_$clinear main_l1) (id `cast` <Co:41>)) `cast` <Co:42>) t_val of { I# x_a2c8 -> case ((($fStrongForget_$clinear main_l2) (id `cast` <Co:41>)) `cast` <Co:42>) t_val of { I# y_a2cb -> case $wshowSignedInt 0# (+# x_a2c8 y_a2cb) [] of { (# ww5_a277, ww6_a278 #) -> : ww5_a277 ww6_a278 } } } So linear doesn't get inlined even though it's a small function that is even explicitly marked INLINE<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.co…>. This prevents further optimizations from happening and leaves optics-related code in semi-optimized state, making looking at cost centers unreliable (see well-typed/optics#324<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.co…>). When profiling is disabled, everything inlines and optimizes away as expected: main1 = case t_val of { T ds_d25F ds1_d25G -> case ds_d25F of { I# x_a2c7 -> case ds1_d25G of { I# y_a2ca -> case $wshowSignedInt 0# (+# x_a2c7 y_a2ca) [] of { (# ww5_a276, ww6_a277 #) -> : ww5_a276 ww6_a277 } } } } I'm attaching archive with cabal project that contains above module for easy reproduction: prof_test.tar.gz<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.ha…> Profiling build was tested with profiling: True in cabal.project.local. Expected behavior GHC should inline class methods marked as INLINE when profiling is enabled. Environment * GHC version used: 8.10.1 Optional: * Operating System: Arch Linux * System Architecture: x86_64 — Reply to this email directly or view it on GitLab<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.ha…>. You're receiving this email because of your account on gitlab.haskell.org. If you'd like to receive fewer emails, you can unsubscribe<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.ha…> from this thread or adjust your notification settings.

1 0