- ghc-devs - Haskell.org

Changes to performance testing?
by Ben Gamari 23 Feb '21

23 Feb '21

Hi all, Recently our performance tests have been causing quite some pain. One reason for this is due to our new Darwin runners (see #19025), which (surprisingly) differ significantly in their performance characteristics (perhaps due to running Big Sur or using native tools provided by nix?). However, this is further exacerbated by the fact that there are quite a few people working on compiler performance currently (horray!). This leads to the following failure mode during Marge jobs: 1. Merge request A improves test T1234 by 0.5%, which is within the test's acceptance window and therefore CI passes 2. Merge request B *also* improves test T1234 by another 0.5%, which similarly passes CI 3. Marge tries to merge MRs A and B in a batch but finds that the combined 1% improvement in T1234 is *outside* the acceptance window. Consequently, the batch fails. This is quite painful, especially given that it creates work for those trying to improve GHC (as the saying goes: no good deed goes unpunished). To mitigate this I would suggest that we allow performance test failures in marge-bot pipelines. A slightly weaker variant of this idea would instead only allow performance *improvements*. I suspect the latter would get most of the benefit, while eliminating the possibility that a large regression goes unnoticed. Thoughts? Cheers, - Ben

4 4

On CI
by Moritz Angermann 22 Feb '21

22 Feb '21

Friends, I've been looking at CI recently again, as I was facing CI turnaround times of 9-12hs; and this just keeps dragging out and making progress hard. The pending pipeline currently has 2 darwin, and 15 windows builds waiting. Windows builds on average take ~220minutes. We have five builders, so we can expect this queue to be done in ~660 minutes assuming perfect scheduling and good performance. That is 11hs! The next windows build can be started in 11hs. Please check my math and tell me I'm wrong! If you submit a MR today, with some luck, you'll be able to know if it will be mergeable some time tomorrow. At which point you can assign it to marge, and marge, if you are lucky and the set of patches she tries to merge together is mergeable, will merge you work into master probably some time on Friday. If a job fails, well you have to start over again. What are our options here? Ben has been pretty clear about not wanting a broken commit for windows to end up in the tree, and I'm there with him. Cheers, Moritz

8 15

Weekly show & tell video meeting
by Csaba Hruska 16 Feb '21

16 Feb '21

Hello, Would you be interested in a weekly show & tell video meeting? The topic would be Haskell & compilers in general. Either GHC or non-GHC related. Regards, Csaba

2 3

cep_app traces
by Simon Peyton Jones 15 Feb '21

15 Feb '21

Ben I'm getting a lot of cpe_app(keepAlive#) 3 cpe_app(keepAlive) trace messages from HEAD. Maybe it's a leftover from tracing when you were developing? Remove? Simon

2 1

gitlab.haskell.org certificate expired?
by Richard Eisenberg 15 Feb '21

15 Feb '21

Hi Ben, It looks like the Let's Encrypt certificate for gitlab.haskell.org <http://gitlab.haskell.org/> has expired, as of about 15 minutes ago. I guess it's time to renew. Thanks, Richard

5 4

Slowing down churn in master prior to 9.2 freeze
by Ben Gamari 13 Feb '21

13 Feb '21

Hi all, Since we are in the final weeks of preparation before the 9.2 branch it would be appreciated if we could refrain from pushing work that isn't either a bug-fix or release-critical feature starting next week, especially where the work requires coordination across submodules. At the moment we have quite a backlog in the merge queue, with a number of dependent patches touching submodules. Consequently, I am a tad worried that the three weeks of runway that we currently have may prove to be insufficient unless we try to keep the merge traffic down. I suggest that we set the milestone of non-release-critical patches to %9.4.1 instead of assigning @marge-bot to allow merging to easily proceed after the branch is cut. Cheers, - Ben

1 0

Boot libraries for GHC 9.2
by Ben Gamari 12 Feb '21

12 Feb '21

Hi all, I am currently in the process of planning out the GHC 9.2 release. As you know, part of this process is working out which core library versions we will ship. My current plan is the following: package maintainer version --------------- ------------------ ----------------- Cabal Mikhail Glushenkov 3.6.0.0 Win32 Tamar Christina 2.10.0.0 array GHC HQ 0.5.3.0 binary Lennart Kolmodin 0.8.7.0 bytestring Andrew Lelechenko 0.11.0.0 (see GHC #19091) containers David Feuer/CLC? 0.6.4.1 deepseq CLC 1.4.4.0 directory CLC 1.3.6.1 exceptions Ed Kmett 0.10.4 filepath Neil Mitchell? 1.4.2.1 haddock Hécate ? haskeline Judah Jacobson 0.8.1.0 hsc2hs GHC HQ 0.68.7 mtl Ed Kmett 2.2.2 parsec Herbert Valerio Reidel 3.1.14 pretty David Terei? 1.1.3.6 process Michael Snoyman 1.6.10.0 stm GHC HQ 2.5.0.0 terminfo Judah Jacobson 0.4.1.4 text CLC 1.2.4.0 time Ashley Yakeley 1.11.1 transformers Ross Paterson 0.5.6.2 unix Andrew Lelechenko 2.8.0.0? xhtml Chris Dornan 3000.2.2.1 If you disagree with this plan or see a question mark next to a library that you are responsible for, please do be in touch. Cheers, - Ben

1 0

Re: Plan for GHC 9.2
by Roland Senn 12 Feb '21

12 Feb '21

I hope ticket #19157 will make it in the GHC 9.2 release. In the GHCi debugger it adds the possibility to set ignore counts to breakpoints. The next <count> times the break point is reached the program's execution does not stop. This feature is available in nearly every debugger, but until now not yet in the GHCi debugger. Merge request !4839 is ready for review (and it's NOT rocket science...)

4 4

Stop holding hadrian back with backwards compatibility
by Moritz Angermann 11 Feb '21

11 Feb '21

Hi, so we've finally run into a case where we need to bump the rts version. This has a great ripple effect. There is some implicit assumption that rts-1.0 will always be true. Of course that was a lie, but a lie we lived with for a long time. Now, hadrian tries *really* hard to replicate some of the Make based build systems idiosyncrasies, this includes creating versionless symlinks for the rts. E.g. libHSrts<X> -> libHSrts-1.0<X>. There is a great deal of logic just to achieve this, and of course it all crumbles now. I'd therefore like to float and propose the idea that we agree to *not* bother (too?) much with make based build systems backwards compatibility and warts that grew over the years in the make based build system with hadrian going forward. Yes, I can probably fix this, and add even more code to this burning pile of complexity, but why? The next person will assume libHSrts does not need to be versioned and continue with this mess. Let's have Hadrian be a clean cut in some areas (it already is, it does away with the horrible abomination that ghc-cabal is--which only serves the purpose of translating cabal descriptions into make readable files), and not be bogged down by backwards compatibility. This is thus my call for voicing concern or the upkeep of legacy support, or I'll take silence as the collective support of making hadrian *not* be held back by backwards compatibility. (This would mean in this case, that I'd just delete the backwards compat code instead of adding even more to it). I hope we all still want Hadrian to replace Make, if not and we want to keep Make, why are we concerning ourselves with Hadrian in the first place. If we are intending to ditch Make, let's not be held back by it. Cheers, Moritz

4 6

Newtype over (~#)
by Igor Popov 10 Feb '21

10 Feb '21

Hello list! Recently I've had this idea of letting Haskell source handle unboxed equalities (~#) by the means of an unboxed newtype. The idea is pretty straightforward: in Core Constraint is Type and (=>) is (->), so a datatype like data Eq a b = (a ~ b) => Refl could become a newtype because it only has a single "field": (a ~ b). Furthermore now that we have unlifted newtypes, we could write it as a newtype over (~#), of kind TYPE (TupleRep []). Defining such a datatype is of course impossible in source Haskell, but what I came up with is declaring a plugin that magically injects the necessary bits into the interface file. Sounds like it should be straightforward: define a newtype (:~:#) with a constructor whose representation type is: forall k (a :: k) (b :: k). (a ~# b) -> a :~:# b The worker constructor is implemented by a coercion (can even be eta-reduced): axiom N::~:# :: forall {k}. (:~:#) = (~#) Refl# = \ (@ k) (@ (a :: k)) (@ (b :: k)) (v :: a ~# b) -> v `cast` (Sym (N::~:#) <k>_N) <a>_N _N And the wrapper constructor has a Haskell-ish type: $WRefl# :: forall k (b :: k). b :~:# b $WRefl# = \ (@ k) (@ (b :: k)) -> Refl# @ k @ a @ b @~ _N Caveat: we don't actually get to specify the unwrappings ourselves, and we have to rely on mkDataConRep generating the right wrapper based on the types and the EqSpec (because this will have to be done every time the constructor is loaded from an iface). In this case the machinery is not convinced that a wrapper is required, unless we ensure that dataConUserTyVarsArePermuted by fiddling around with ordering of variables. This is a minor issue (which I think I can work around) but could be addressed on the GHC side. I've indeed implemented a plugin that declares these, but we run into a more major issue: the simplifier is not ready for such code! Consider a simple utility function: sym# :: a :~:# b -> b :~:# a sym# Refl# = Refl# This gets compiled into: sym# = \ (@ k) (@ (a :: k)) (@ (b :: k)) (ds :: a :~:# b) -> case ds `cast` N::~:# <k>_N <a>_N _N of co -> $WRefl# @ k @ b `cast` ((:~:#) <k>_N (Sym co))_R which seems valid but then the simplifier incorrectly inlines the unfolding of $WRefl# and arrives at code that is not even well-scoped (-dcore-lint catches this): sym# = \ (@ k) (@ (a :: k)) (@ (b :: k)) (ds :: a :~:# b) -> case ds `cast` N::~:# <k>_N <a>_N _N of co -> v `cast` (Sym (N::~: <k>_N)) _N _N ; ((:~:#) <k>_N _N (Sym co))_R Actually the problem manifests itself even earlier: when creating an unfolding for the wrapper constructor with mkCompulsoryUnfolding we run the unfolding term through simpleOptExpr once before memorizing the unfolding, and this produces an unfolding for $WRefl# that is broken (ill-scoped) in a similar fashion: $WRefl = \ (@ k) (@ (b :: k)) -> v `cast` Sym (N::~:# <k>_N) _N _N And we can verify that the issue is localized here: applying simpleOptExpr to: (\ (v :: a ~# a) -> v `cast` <a ~# a>_R) @~ <a>_N results in: v The former term is closed and the latter is not. There is an invariant on mkPrimEqPred (though oddly not on mkReprPrimEqPred) that says that the related types must not be coercions (we're kind of violating this here). I have several questions here: - Is there a good reason for the restriction that equalities should not contain other equalities? - Should this use case be supported? Coercions are almost first class citizens in Core (are they?), makes sense to let source Haskell have a bit of that? - Does it make sense to include this and a few similar types (unboxed Coercion and Dict) as a wired in type packaged with GHC in some form? -- mniip

3 6