Relatedly, there is a sketch of something like a "case construct" on Richard's blog, where he talks about a "type instance where" syntax that allows closed interdependent sets of family instances. Is that something that was later found to be infeasible? On Fri, May 12, 2017, 17:53 wrote:

Send Haskell-Cafe mailing list submissions to haskell-cafe@haskell.org

To subscribe or unsubscribe via the World Wide Web, visit http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe or, via email, send a message with subject or body 'help' to haskell-cafe-request@haskell.org

You can reach the person managing the list at haskell-cafe-owner@haskell.org

When replying, please edit your Subject line so it is more specific than "Re: Contents of Haskell-Cafe digest..." Today's Topics:

1. Re: Fast JSON validation - reducing allocations (Arjen) 2. Re: Fast JSON validation - reducing allocations (David Turner) 3. Re: [ghc-proposals/cafe] Partially applied type families (Anthony Clayden)

---------- Forwarded message ---------- From: Arjen To: David Turner , Haskell Cafe < haskell-cafe@haskell.org> Cc: Bcc: Date: Fri, 12 May 2017 12:16:45 +0200 Subject: Re: [Haskell-cafe] Fast JSON validation - reducing allocations On Fri, 2017-05-12 at 10:48 +0100, David Turner wrote:

...

On 12 May 2017 at 09:27, Arjen wrote:

...

Maybe this is a silly question, and please let me know why if so, but:

Has anyone thought about parallelizing it for multiple messages in order to "produce garbage faster"? While reducing allocation will make the single validations faster, doing multiple ones might improve the throughput per GC ratio. This assumes that the amount of live data in the heap is small, making GC sort of constant time, and having multiple cores available.

Not a silly question at all. Adding the following incantation:

`using` parListChunk 100 rseq

does quite happily spread things across all 4 cores on my development machine, and it's certainly a bit faster. To give some stats, it processes ~24 events between GCs rather than ~6, and collects ~2MB rather than ~500kB. The throughput becomes a lot less consistent, however, at least partly due to some bigger GC pauses along the way. As Ben's statistics showed, our allocation rate on one thread is around 4TBps, which is already stressing the GC out a bit, and parallelising it doesn't make that problem any easier.

I know in the OP I said "we have a stream" (accidental MLK misquote) but in fact there are a bunch of parallel streams whose number exceeds the number of available cores, so we don't anticipate any enormous benefit from spreading the processing of any one stream across multiple cores: single-threaded performance is what we think we should be concentrating on.

Cheers,

David

Apologies for spamming, but if you want more mutator time between garbage collections, try increasing the nursery size. I get a lot less time in GC and also much less sync's on GC (according to -s). It seems to reduce the execution time by a third using something -A8M or -A64M (quite extreem). In a large program, this effect might be less.

kind regards, Arjen

---------- Forwarded message ---------- From: David Turner To: Arjen Cc: Haskell Cafe Bcc: Date: Fri, 12 May 2017 11:24:23 +0100 Subject: Re: [Haskell-cafe] Fast JSON validation - reducing allocations On 12 May 2017 at 11:00, Arjen wrote:

...

Happy to read that you have more success than I with parListChunk. I expect(ed) the sparks to be cheap, so this might help if the various incoming streams are very unbalanced in length and/or message sizes. I agree that multi-processing a single stream does not make much sense if you already have multiple concurrent streams.

Nice to see that adding on parallelism in Haskell (GHC) is that easy (in this case) and with a very good speed-up factor!

kind regards, Arjen

Ah, ok, this is strange. The production system is definitely faster with the parallel strategy (on one stream) but my Criterion benchmarks are much slower. I've updated the code and results on Github ( https://github.com/DaveCTurner/json-validator). Probably not going to be able to look at the parallel version much more, but thought you might like a look.

Cheers,

---------- Forwarded message ---------- From: Anthony Clayden To: haskell-cafe@haskell.org Cc: Bcc: Date: Fri, 12 May 2017 22:46:45 +1200 Subject: Re: [Haskell-cafe] [ghc-proposals/cafe] Partially applied type families

...

2 days ago Richard Eisenberg (and mniip) wrote:

...

... On incremental improvements / the "half-baked" nature of type families: I agree completely. ... ["half-baked" is Richard quoting mniip]

...

... the weird place of type families in the language. It is my hope that Dependent Haskell will obviate a lot of these concerns, allowing us to deprecate type families as we know them (you would just use an ordinary function in types instead).

I don't want to sidetrack the github discussion on mniip's proposal. https://github.com/ghc-proposals/ghc-proposals/pull/52

So please explain here in what way type families are "half baked".

The part of type families I find hard to use and hard to reason about is closed type families. I'd be very happy to deprecate them, if there was something that did overlaps better.

...

From what I can make out, though, in Richard's Dissertation, they can't be promoted to "ordinary functions in types".

If the closed equations were ordinary functions in types, you could go:

...

type family F a b type instance F Int b = b type instance F a Bool | a /~ Int = a -- non-unifiability guard

With no need for a closed sequence of choices.

Is this the "case-like computations" others talk about on the proposal?

AntC

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe