Re: FW: Lightweight Concurrency Branch

Hi KC, if blackholes only appear during thunk evaluation, could the problem you describe below be worked around by simply imposing that the scheduler never creates black holes? Say by leveraging GHC's new -XStrict language extension? -- Mathieu Boespflug Founder at http://tweag.io. On 4 January 2017 at 13:54, Simon Peyton Jones via ghc-devs < ghc-devs@haskell.org> wrote:

Reply from KC… see below.

Simon

*From:* KC Sivaramakrishnan [mailto:sk826@cam.ac.uk] *Sent:* 04 January 2017 11:29 *To:* Simon Peyton Jones *Cc:* Daniel Bennet ; ghc-devs@haskell.org; KC (sk826@hermes.cam.ac.uk) *Subject:* Re: Lightweight Concurrency Branch

Hi Simon, David,

Indeed as Simon mentioned it, the work was never finished. The implementation is(was) at a stage where we can run some small non-trivial benchmarks (Section 7 of the JFP paper). The interactions with FFI, MVars, STM, asynchronous exceptions worked well (though we most probably do things a little differently now), until we encountered interaction with blackholes mechanism.

The crux of the problem is that the blackholing mechanism interacts with the scheduler and if the scheduler functionality itself is written in Haskell, then we have the potential for a deadlock. Some of the details are presented in Section 6.5, but we never got around to a clean solution that I could get working properly. Hence the reason for not formalizing it completely in the paper, and I am sure there are some edge cases that I hadn't thought about. This is a particularly tricky issue, and the current design unfortunately does not lend itself to a clean solution.

Although the asynchronous exception mechanism interaction was not formalized, it works well and passes the tests. Not much effort was put in to make the implementation go particularly fast. While the performance was comparable on average, but varied quite a bit on edge cases; some of the results in the paper show this.

If one were to revive the project, I would suggest starting from the design, using the existing code as a prototype, but write code from scratch; pleasantly there isn't much new code in this branch. The project does need substantial amount of work to make it upstream with the newer RTS mechanisms. I am very happy to provide more details and eager to assist with the work, but my time commitments mean that I cannot lead this effort.

Kind Regards,

KC

On Wed, Jan 4, 2017 at 10:38 AM, Simon Peyton Jones wrote:

David

KC never finished work on this stuff. I’m copying him because I’m sure he’d be happy to help.

KC: can you summarise where you left it?

I think it’s very interesting work, and has the potential to make GHC’s RTS much more malleable, by moving more of it into Haskell libraries instead of deeply-magic C code.

But it’s not easy, because we are reluctant to lose performance, and because there are interactions with STM, weak pointers, foreign function calls, etc. I think it’d require a bit of commitment to make a go of it.

Simon

*From:* ghc-devs [mailto:ghc-devs-bounces@haskell.org] *On Behalf Of *Daniel Bennet *Sent:* 28 December 2016 17:23 *To:* ghc-devs@haskell.org *Subject:* Lightweight Concurrency Branch

The lightweight concurrency branch is highly interesting and relevant to my interests, however, the ghc-lwc2 branch hasn't been updated in several years even though it's listed as an active branch at https://ghc.haskell.org/trac/ghc/wiki/ActiveBranches

The wiki page for the work hasn't been updated in almost two years either, https://ghc.haskell.org/trac/ghc/wiki/LightweightConcurrency

Relevant papers:

Composable Scheduler Activations for Haskell (2014)

https://timharris.uk/papers/2014-composable-tr.pdf https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftimharris.uk%2Fpapers%2F2014-composable-tr.pdf&data=02%7C01%7Csimonpj%40microsoft.com%7Cd132a48641a54f732ffc08d42f463e8b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636185426136080161&sdata=YG%2B0xfi%2Bd74Vi9GBI8iTrr4GEMzjS2WwrI182%2BcEgqM%3D&reserved=0

Composable Scheduler Activations for Haskell (2016)

http://kcsrk.info/papers/schedact_jfp16.pdf https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fkcsrk.info%2Fpapers%2Fschedact_jfp16.pdf&data=02%7C01%7Csimonpj%40microsoft.com%7Cd132a48641a54f732ffc08d42f463e8b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636185426136080161&sdata=mH79Y3U%2BdDvcw2PA8lGcFaPMJw6DgCIXbu%2F3WGvyGaU%3D&reserved=0

What remains for integrating this branch into GHC?

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs