- ghc-devs - Haskell.org

GHC Test Failures
by Gert-Jan Bottu 10 Feb '20

10 Feb '20

Hi all, It's my first time working on ghc (currently implementing proposal 99 [1]), and I have two issues with the ghc tests that I can't seem to figure out: - Running hadrian/build.sh test -j on my machine causes 78 tests to fail with the following error: collect2: error: ld returned 1 exit status `cc' failed in phase `Linker'. (Exit code: 1) This happens in both my wip branch and on master, both in devel2 and quickest flavour. I've tried hadrian/build.sh clean and even recloned the repo, but the errors are quite persistent. Running the hadrian tests on CI works fine though. - On CI [2], all tests under build pass. Under full-build however, the hadrian tests pass, but a single make test fails : Wrong exit code for haddockHtmlTest()(expected 0 , actual 2 ) Stdout ( haddockHtmlTest ): [1 of 6] Compiling Test.Haddock.Process ( /builds/Gertjan423/ghc/testsuite/../utils/haddock/haddock-test/src/Test/Haddock/Process.hs, Test/Haddock/Process.o ) [2 of 6] Compiling Test.Haddock.Utils ( /builds/Gertjan423/ghc/testsuite/../utils/haddock/haddock-test/src/Test/Haddock/Utils.hs, Test/Haddock/Utils.o ) [3 of 6] Compiling Test.Haddock.Config ( /builds/Gertjan423/ghc/testsuite/../utils/haddock/haddock-test/src/Test/Haddock/Config.hs, Test/Haddock/Config.o ) [4 of 6] Compiling Test.Haddock ( /builds/Gertjan423/ghc/testsuite/../utils/haddock/haddock-test/src/Test/Haddock.hs, Test/Haddock.o ) [5 of 6] Compiling Test.Haddock.Xhtml ( /builds/Gertjan423/ghc/testsuite/../utils/haddock/haddock-test/src/Test/Haddock/Xhtml.hs, Test/Haddock/Xhtml.o ) [6 of 6] Compiling Main ( /builds/Gertjan423/ghc/testsuite/../utils/haddock/html-test/Main.hs, Main.o ) Linking html-test ... Haddock version 2.22.0, (c) Simon Marlow 2006 Ported to use the GHC API by David Waern 2006-2008 8.11.0.20200207 Generating documentation... Failed to run Haddock on test package '' Haddock output is at 'haddock-out.log'. This file can be set with `--haddock-stdout`. Testing output files... Some tests crashed. Makefile:19: recipe for target 'htmlTest' failed Stderr ( haddockHtmlTest ): Warning: Unknown/unsupported 'ghc' version detected (Cabal 3.2.0.0 supports 'ghc' version < 8.10): /builds/Gertjan423/ghc/inplace/bin/ghc-stage2 is version 8.11.0.20200207 make[2]: *** [htmlTest] Error 1 *** unexpected failure for haddockHtmlTest(normal) I did make some small Haddock changes, so I did most likely cause this behavior. I just find it strange that this error does not happen on my machine, and even on CI only occurs in the make tests (and only under linux, not under windows). Any tips regarding what could cause this, and how to tackle debugging this would be greatly appreciated :) Cheers, Gert-Jan [1] : https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0099-e… [2] : https://gitlab.haskell.org/Gertjan423/ghc/pipelines/15509

1 0

Free dictionary variables in elaborated core expressions
by Yiyun Liu 03 Feb '20

03 Feb '20

Hi ghc-devs, About 10 days ago, I made a thread about defining a function called elaborateExpr which turns a string into a core expression <https://github.com/yiyunliu/ghc-elaboration-test/blob/8f362ad92dc6601b4cb7e…> within an interactive context. Now here's an unexpected behavior which I'm not sure how to deal with. Given the expression: (\x -> x + 1) :: Int -> Int I expect to get something that looks like: \ (x :: Int) -> + @ Int GHC.Num.$fNumInt x (GHC.Types.I# 1#) where GHC.Num.$fNumInt is the exported dictionary. What I actually get is something like this: \ (x :: Int) -> + @ Int $dNum_someuniqueid x (GHC.Types.I# 1#) where $dNum_someuniqueid is a free dictionary variable within the expression. I was confused by the occurrence of the free variable $dNum at first, but after running the command: "ghc -ddump-ds-preopt somefile.hs" to dump the core bindings, I found that the dictionary variables like $dNum_ are actually local variables defined at the top-level. My objective is to inline those top-level dictionary definitions into the core expression using let bindings, but it seems tricky since I'm doing everything within an interactive context. Calling getBindings <https://hackage.haskell.org/package/ghc-8.6.5/docs/GHC.html#v:getBindings> only gives me the expression I elaborated, but the dictionary is no where to be found. Interestingly, when I turn on flags such as "DeferTypedHoles" or "DeferOutOfScopeVariables", all the dictionaries are defined locally in let bindings. However, I can't replicate that behavior even with the flags on in the interactive context. How do I find the dictionaries? Thanks, - Yiyun

2 2

Allowing Windows CI to fail
by Ben Gamari 03 Feb '20

03 Feb '20

Hi everyone, After multiple weeks of effort struggling to get Windows CI into a stable condition I'm sorry to say that we're going to need to revert to allowing it to fail for a bit longer. The status quo is essentially holding up the entire merge queue and we still seem quite far from resolving the issues. I have summarised the current state-of-play in #17777. In short, the gcc toolchain likely can't be used reliably on Windows due to its ubiquitous use of `exec`, which cannot be reliably implemented on Windows. Switching to LLVM as our native toolchain was my (initially promising) last-ditch attempt at avoiding this issue but sadly this looks to be a long road. My current attempt is stuck on an inscrutable loader error. For the short-term, I am afraid I have run out of time for this effort. My current plan is to merge what I can from my wip/windows-ci branch but again enable the Windows CI jobs' allow_failure flag so that its unreliable nature doesn't hold up otherwise-passing CI jobs. While it's unfortunately that we still lack reliable CI on Windows, I think the effords of the last few weeks were quite worthwhile. We now have: * A much better understanding of the issues affecting us on Windows * Significantly better documentation and automation for producing our mingw toolchain artifacts * better scripting for setting up Windows CI runners * fixed several bugs in the ghc-jailbreak library used to work around the Windows MAX_PATH limitation Many thanks to Tamar Christina for his many hours of patient help. Without him, GHC's Windows support would be in significantly worse shape than it is. Users of GHC should note that the CI issues we are struggling with *do not* affect compiled code. These bugs manifest only as (rare) failed compilations (particularly when building GHC itself); however, once compilation succeeds the program that results is correct and reliable. Cheers, - Ben

2 2

testing ... ping
by Compl Yue 03 Feb '20

03 Feb '20

Sorry if this bothered you, I have problem sending email to the list, sending this for bgamari to further diagnostic the situation.

1 0

Random crashes with memory corruption symptoms
by Harendra Kumar 03 Feb '20

03 Feb '20

Hi, While running a test-suite for the streaming library streamly I am encountering a crash which seems to happen at random places at different times. The common messages are: * Segmentation fault: 11 * internal error: scavenge_mark_stack: unimplemented/strange closure type 24792696 @ 0x4200a623e0 * internal error: update_fwd: unknown/strange object 223743520 and several other such messages. Prima facie this looks like the memory is getting corrupted/scribbled somehow. My first suspicion was that this could be a problem in the streamly library code. But I have stripped down the code to bare minimum and there is no C FFI code or no poking to memory pointers. My next suspicion was the hspec/quickcheck testing code that is being used in this test. I checked the hspec code to ensure that there is no C code/pointer poking in any of the code involved. But no luck there as well, still looking to further strip down that code. My suspicion now is moving more towards the GHC RTS. This issue only shows when the following conditions are met: * hspec "parallel" combinator is used to run tests in parallel * streamly concurrent code is being tested which can create many threads * The GHC heap size is restricted to a small size ~32MB using "-M32M" rts option. * It is consistently seen with GHC 8.6.5 as well as GHC 8.8.1 It never occurs when the heap size is not restricted. I have seen random crashes before as well with a "IO manager die" message, when using concurrent networking IO with streamly. Though earlier it was not easily reproducible, I stopped chasing it. But now it looks like that issue might also be a manifestation of the same underlying problem. My guess is it could be something in the RTS concurrency/threading related code. Let me know if the symptoms ring a bell or if you can point to something specific based on the symptoms. Also, what are the usual tools/methods/debugging aids/flags to debug such issues in GHC? If not a GHC issue what are the possible ways in which such problem can be induced by application code? Meanwhile, I am also trying to simplify the reproducing code further to remove other factors as much as possible. The current code is at https://github.com/composewell/streamly on the ghc-segfault branch. Run "$ while true; do cabal run properties || break; done" in the shell and if you are lucky it may crash soon. The test code is in "test/Prop.hs" - here https://github.com/composewell/streamly/blob/ghc-segfault/test/Prop.hs . -harendra

3 6

RE: Simplifying subsumption
by Simon Peyton Jones 03 Feb '20

03 Feb '20

Jurriaan, It would be good to dig out the patches that affect packages without which GHC cannot build. (E.g. haskeline.) Would that take long? You have them already... It doesn't have to be perfect or git-tracked; just send a patch file in an email. I'm just trying to avoid re-inventing the wheel. A more systematic set of patches, for all the hackage packages you looked at, could certainly wait a week or three. Thanks Simon | -----Original Message----- | From: Jurriaan Hage <J.Hage(a)uu.nl> | Sent: 02 February 2020 19:00 | To: Simon Peyton Jones <simonpj(a)microsoft.com> | Cc: Jurriaan Hage <jur(a)cs.uu.nl>; Ben Gamari <ben(a)well-typed.com>; ghc- | devs <ghc-devs(a)haskell.org> | Subject: Re: Simplifying subsumption | | Hi Simon, | | Fine by me. | What kind of timeline are we looking at? I have to grade exams the coming | week, since I was ill last week, | and the week afterwards I am in Poland (but maybe I do have time for this | kind of thing then). | | Jur | | > On 2Feb, 2020, at 19:38, Simon Peyton Jones <simonpj(a)microsoft.com> | wrote: | > | > Jurriaan, Ben | > | > I have a MR up for #17775, which implements the now-accepted GHC | proposal to simplify subsumption. | > | > | https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.h | askell.org%2Fghc%2Fghc%2Fmerge_requests%2F2600&data=02%7C01%7Csimonpj% | 40microsoft.com%7C3d05874428304c245e9d08d7a8121882%7C72f988bf86f141af91ab2 | d7cd011db47%7C1%7C0%7C637162667999082070&sdata=Y7s3L%2F1xrCSxdF0EfRNik | YJnOU2Y6RtVlR92irMg7cU%3D&reserved=0 | > | > However, it requires some eta expansions in haskeline, and maybe other | packages. I don’t know how to do that. Could you, or another member of | the GHC support team, help do this? We want a MR we can validate on its | own before committing, of course. | > | > Jurriaan has all the necessary library patches. | > | > We should not commit to HEAD until there has been some code review, | though. | > | > Moreover as we know, when committed, it’ll break a number of packages, | so we need to broadcast a warning, along with the patches Jurriaan has to | fix them. I’m not sure when and how to do that. | > | > Thanks! | > | > Simon | >

1 0

Simplifying subsumption
by Simon Peyton Jones 02 Feb '20

02 Feb '20

Jurriaan, Ben I have a MR up for #17775, which implements the now-accepted GHC proposal to simplify subsumption. https://gitlab.haskell.org/ghc/ghc/merge_requests/2600 However, it requires some eta expansions in haskeline, and maybe other packages. I don't know how to do that. Could you, or another member of the GHC support team, help do this? We want a MR we can validate on its own before committing, of course. Jurriaan has all the necessary library patches. We should not commit to HEAD until there has been some code review, though. Moreover as we know, when committed, it'll break a number of packages, so we need to broadcast a warning, along with the patches Jurriaan has to fix them. I'm not sure when and how to do that. Thanks! Simon

1 0

How to run the testsuite with an out of tree compiler?
by Matthew Pickering 01 Feb '20

01 Feb '20

I am trying to run `ghcide` on GHC's tests. The documentation says that I should pass `--test-compiler=..` to hadrian in order to do this. When I do this however it starts trying to configure a bunch of packages and starting to build stuff. I don't want hadrian to build anything, I just want it to use my patched version of GHC when running certain tests! How can I achieve this? Cheers, Matt

1 1

[ANNOUNCE] Glasgow Haskell Compiler 8.10.1-rc1 released
by Ben Gamari 31 Jan '20

31 Jan '20

Hello all, The GHC team is happy to announce the availability of the first release candidate of GHC 8.10.1. Source and binary distributions are available at the usual place: https://downloads.haskell.org/ghc/8.10.1-rc1/ GHC 8.10.1 will bring a number of new features including: * The new UnliftedNewtypes extension allowing newtypes around unlifted types. * The new StandaloneKindSignatures extension allows users to give top-level kind signatures to type, type family, and class declarations. * A new warning, -Wderiving-defaults, to draw attention to ambiguous deriving clauses * A number of improvements in code generation, including changes * A new GHCi command, :instances, for listing the class instances available for a type. * An upgraded Windows toolchain lifting the MAX_PATH limitation * A new, low-latency garbage collector. * Improved support profiling, including support for sending profiler samples to the eventlog, allowing correlation between the profile and other program events This is the first and likely final release candidate. For a variety of reasons, it comes a few weeks later than the original schedule of release late December. However, besides a few core libraries book-keeping issues this candidate is believed to be in good condition for the final release. As such, the final 8.10.1 release will likely come in two weeks. Note that at the moment we still require that macOS Catalina users exempt the binary distribution from the notarization requirement by running `xattr -cr .` on the unpacked tree before running `make install`. In addition, we are still looking for any Alpine Linux to help diagnose the correctness issues in the Alpine binary distribution [1]. If you use Alpine any you can offer here would be greatly appreciated. Please do test this release and let us know if you encounter any other issues. Cheers, - Ben [1] https://gitlab.haskell.org/ghc/ghc/issues/17508 [2] https://gitlab.haskell.org/ghc/ghc/issues/17418

5 4

more submodule questions
by Richard Eisenberg 30 Jan '20

30 Jan '20

Hi devs, I recently found this text at the end of https://gitlab.haskell.org/ghc/ghc/wikis/working-conventions/git/submodules <https://gitlab.haskell.org/ghc/ghc/wikis/working-conventions/git/submodules> : --- The CI pipeline of ghc/ghc> <applewebdata://9E3625B8-2065-48C6-A8FA-16817EA6E642/ghc/ghc> includes a linting step to ensure that all submodules refer only to "persistent" commits of the upstream repositories (e.g. not wip/ branches, which may disappear in the future). Specifically, the linter checks that any submodules refer to commits that are reachable by at least one branch that doesn't begin with the prefix wip/. Consequently, you must ensure that any submodule changes introduced in a ghc/ghc> <applewebdata://9E3625B8-2065-48C6-A8FA-16817EA6E642/ghc/ghc> merge request are merged upstream before the merge request is added to the merge queue. --- I don't understand what this means. - By citing "ghc/ghc>", does this mean that the linter only checks for this on branches of the ghc/ghc repo? If I have a fork (e.g. rae/ghc), are these checks disabled? - Does this linter stop CI from progressing to, say, running the testsuite? If so, then how can we run the testsuite via CI if we have any submodule changes? We want to run the testsuite while the work is still in progress. - By "you must ensure ... before the merge request is added to the merge queue": this makes me wonder whether the linter is just a warning or an error. That is, if I must ensure it, then it suggests that CI is not ensuring it. Sorry to be dense here! Thanks, Richard

2 3