>These two assumptions are basically the most challenging issues stop people from getting more performance

I guess so, that is why I've decided to introduce it. Actually this design was inspired by my job task. I'm working on graph processing library written in Scala, which stores large amount of data in of-heap memory in binary format. I want to read this data. Storable is not good enough because there is a variable-length data types as well. On other hand all always available loaded in memory and no incremental parsing required.

So, my approach is something between Storable and fully featured serializes such as binary/cereal.

ср, 20 мар. 2019 г. в 09:20, Dr.Koster <drkoster@qq.com>:

These two assumptions are basically the most challenging issues stop people from getting more performance, e.g. If somehow you want to support incremental parsing, a primitive like `ensureN` probably need to do something like

```
ensureN n = Parser $ \ buf ... k ->
if n < length buf then k buf ...
else partial buf ... k
```

The k appear twice inside `ensureN` 's both branch, which will stop parsers from getting inline sooner or later since inlining k into both branch will produce exponential amount of code, so instead GHC will allocate k and make a jump, the best we can get is probably to try make it a joint point, so no allocation is needed, but that's the best you can get from argument passing buffers.

Regards
Han Dong

------------------ 原始邮件 ------------------
发件人: "Joachim Durchholz"<jo@durchholz.org>;
发送时间: 2019年3月20日(星期三) 中午1:28
收件人: "haskell-cafe"<haskell-cafe@haskell.org>;
主题: Re: [Haskell-cafe] Yet another binary reader (10x faster thanHackage's binary; )

Am 19.03.19 um 22:58 schrieb Станислав Черничкин:
> possibly because of inlining of
> specialization magic.

I have seen people validate this kind of assumption, by looking at the
various intermediate representations of code.
If you're after performance, you may want to do that. You may find that
things work differently than expected, with the potential for becoming a
bottleneck when stuff is scaled up (more varied datatypes in the input
stream, or more complex target datatypes, stuff hidden behind "oh I
don't need GC yet", that kind of finding).

Regards,
Jo
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.