February 2020 - ghc-devs

Random crashes with memory corruption symptoms
by Harendra Kumar 03 Feb '20

03 Feb '20

Hi, While running a test-suite for the streaming library streamly I am encountering a crash which seems to happen at random places at different times. The common messages are: * Segmentation fault: 11 * internal error: scavenge_mark_stack: unimplemented/strange closure type 24792696 @ 0x4200a623e0 * internal error: update_fwd: unknown/strange object 223743520 and several other such messages. Prima facie this looks like the memory is getting corrupted/scribbled somehow. My first suspicion was that this could be a problem in the streamly library code. But I have stripped down the code to bare minimum and there is no C FFI code or no poking to memory pointers. My next suspicion was the hspec/quickcheck testing code that is being used in this test. I checked the hspec code to ensure that there is no C code/pointer poking in any of the code involved. But no luck there as well, still looking to further strip down that code. My suspicion now is moving more towards the GHC RTS. This issue only shows when the following conditions are met: * hspec "parallel" combinator is used to run tests in parallel * streamly concurrent code is being tested which can create many threads * The GHC heap size is restricted to a small size ~32MB using "-M32M" rts option. * It is consistently seen with GHC 8.6.5 as well as GHC 8.8.1 It never occurs when the heap size is not restricted. I have seen random crashes before as well with a "IO manager die" message, when using concurrent networking IO with streamly. Though earlier it was not easily reproducible, I stopped chasing it. But now it looks like that issue might also be a manifestation of the same underlying problem. My guess is it could be something in the RTS concurrency/threading related code. Let me know if the symptoms ring a bell or if you can point to something specific based on the symptoms. Also, what are the usual tools/methods/debugging aids/flags to debug such issues in GHC? If not a GHC issue what are the possible ways in which such problem can be induced by application code? Meanwhile, I am also trying to simplify the reproducing code further to remove other factors as much as possible. The current code is at https://github.com/composewell/streamly on the ghc-segfault branch. Run "$ while true; do cabal run properties || break; done" in the shell and if you are lucky it may crash soon. The test code is in "test/Prop.hs" - here https://github.com/composewell/streamly/blob/ghc-segfault/test/Prop.hs . -harendra

3 6

RE: Simplifying subsumption
by Simon Peyton Jones 03 Feb '20

03 Feb '20

Jurriaan, It would be good to dig out the patches that affect packages without which GHC cannot build. (E.g. haskeline.) Would that take long? You have them already... It doesn't have to be perfect or git-tracked; just send a patch file in an email. I'm just trying to avoid re-inventing the wheel. A more systematic set of patches, for all the hackage packages you looked at, could certainly wait a week or three. Thanks Simon | -----Original Message----- | From: Jurriaan Hage <J.Hage(a)uu.nl> | Sent: 02 February 2020 19:00 | To: Simon Peyton Jones <simonpj(a)microsoft.com> | Cc: Jurriaan Hage <jur(a)cs.uu.nl>; Ben Gamari <ben(a)well-typed.com>; ghc- | devs <ghc-devs(a)haskell.org> | Subject: Re: Simplifying subsumption | | Hi Simon, | | Fine by me. | What kind of timeline are we looking at? I have to grade exams the coming | week, since I was ill last week, | and the week afterwards I am in Poland (but maybe I do have time for this | kind of thing then). | | Jur | | > On 2Feb, 2020, at 19:38, Simon Peyton Jones <simonpj(a)microsoft.com> | wrote: | > | > Jurriaan, Ben | > | > I have a MR up for #17775, which implements the now-accepted GHC | proposal to simplify subsumption. | > | > | https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.h | askell.org%2Fghc%2Fghc%2Fmerge_requests%2F2600&data=02%7C01%7Csimonpj% | 40microsoft.com%7C3d05874428304c245e9d08d7a8121882%7C72f988bf86f141af91ab2 | d7cd011db47%7C1%7C0%7C637162667999082070&sdata=Y7s3L%2F1xrCSxdF0EfRNik | YJnOU2Y6RtVlR92irMg7cU%3D&reserved=0 | > | > However, it requires some eta expansions in haskeline, and maybe other | packages. I don’t know how to do that. Could you, or another member of | the GHC support team, help do this? We want a MR we can validate on its | own before committing, of course. | > | > Jurriaan has all the necessary library patches. | > | > We should not commit to HEAD until there has been some code review, | though. | > | > Moreover as we know, when committed, it’ll break a number of packages, | so we need to broadcast a warning, along with the patches Jurriaan has to | fix them. I’m not sure when and how to do that. | > | > Thanks! | > | > Simon | >

1 0

Simplifying subsumption
by Simon Peyton Jones 02 Feb '20

02 Feb '20

Jurriaan, Ben I have a MR up for #17775, which implements the now-accepted GHC proposal to simplify subsumption. https://gitlab.haskell.org/ghc/ghc/merge_requests/2600 However, it requires some eta expansions in haskeline, and maybe other packages. I don't know how to do that. Could you, or another member of the GHC support team, help do this? We want a MR we can validate on its own before committing, of course. Jurriaan has all the necessary library patches. We should not commit to HEAD until there has been some code review, though. Moreover as we know, when committed, it'll break a number of packages, so we need to broadcast a warning, along with the patches Jurriaan has to fix them. I'm not sure when and how to do that. Thanks! Simon

1 0

How to run the testsuite with an out of tree compiler?
by Matthew Pickering 01 Feb '20

01 Feb '20

I am trying to run `ghcide` on GHC's tests. The documentation says that I should pass `--test-compiler=..` to hadrian in order to do this. When I do this however it starts trying to configure a bunch of packages and starting to build stuff. I don't want hadrian to build anything, I just want it to use my patched version of GHC when running certain tests! How can I achieve this? Cheers, Matt

1 1