- Haskell-Cafe - Haskell.org

[ANN] libarchive
by Vanessa McHale 21 Mar '19

21 Mar '19

Hi all, I just released libarchive (http://hackage.haskell.org/package/libarchive) which can be used to create and unpack tar archives as well as several other formats. It has a reasonably complete high-level API and full bindings to the C library. Cheers, Vanessa McHale

1 0

Yet another binary reader (10x faster than Hackage's binary; )
by Станислав Черничкин 21 Mar '19

21 Mar '19

I was investigating Haskell’s binary serialization libraries and has found that lot of them (binary, cereal and protocol-buffers) has pretty similar design. They capable to consume lazy bytestrings and either produce result, report error or produce partial result, indicating that more input data required. Such design leads to quite complex readers, and often ends up with producing closures in runtime. If you take a close look at binary’s microbenches (binary in general bit faster than cereal, so I focused on binary) you will discover, that it triggers GC even during simple tasks, such as reading a bunch of Word64 and summing results. And it relatively slow. Binary’s performance limited to hundreds of megabytes per second even in simple scenarios, but actual memory bandwidth should allow us process gigabytes per second. So, I decided to try a different approach. My approach is based on two assumptions so if it does not suit your case it obliviously will not work for you. First – when dealing with binary data we always know exactly how many bytes we want to read. This happens because we either read a fixed size primitive type or length-prefixed variable size data (possibly not “prefixed”, but you always know expected message length from somewhere). Second – chunks of data are continuous. I.e. we working with strict bytestrings more often. Even when you processing chunked stream of data with unknown number of messages (such as Mesos RecordIO Format) you should always be able to allocate continuous memory buffer for a single entity you want to deserialize. If it’s ok for you lets go on. First thing I’ve created was so called static reader. Static – because reader knows how mush bytes it wants to consume at compile time. Such readers suitable for reading combinations of primitive values. Here the implementation https://github.com/PROTEINE-INSAIDERS/lev-tolstoy/blob/master/src/Lev/Reade… Static reader is somewhat monadic (you can use do-syntax with RebindableSyntax extension like here https://github.com/PROTEINE-INSAIDERS/lev-tolstoy/blob/master/bench/Bench/L… ) but it actually indexed monad and it unfortunately incompatible with traditional monads. Offsets and sizes are calculates at type level during the compile time and according to microbences static reader does not introduced any overhead compared to hand-written reader using the low-level indexInt64OffAddr# function ( https://github.com/PROTEINE-INSAIDERS/lev-tolstoy/blob/master/bench/Bench/H… ) Next thing was a dynamic reader. Dynamic readers are composed from static readers, they are fully monadic and allow you to read length-prefixed data structures (you read length with wrapped static reader and then perform normal monadic binding to supply a reader which reads a value). Here the implementation https://github.com/PROTEINE-INSAIDERS/lev-tolstoy/blob/master/src/Lev/Reade… I’ve discovered, that typeclasses are surprisingly fast. So, it’s completely ok to have something like “(Cursor c) => … Reader” where cursor contains functions for overflow checking and memory buffer progression. I was not able to achieve same performance when I passed buffer callbacks explicitly, possibly because of inlining of specialization magic. So, I've created ByteStringCursor ( https://github.com/PROTEINE-INSAIDERS/lev-tolstoy/blob/master/src/Lev/Reade… ) which allows to read strict bytestrings. I suppose that cursors introduce flexibility: you may read strict bytestring or you may define custom cursor for reading custom memory buffers. Finally, I’ve written some microbenches for my cases and also ported one microbench from binary. And my implementation was able to run 10x-15x faster than binary on strict bytestring. Moreover, it does not trigger GC during the run at all! You may find the project and all the benches here https://github.com/PROTEINE-INSAIDERS/lev-tolstoy So, what's the purpose of my message? It’s quite unlikely that I’ll release my code on the Hackage. This is because I’m a hobbyist Haskell developer and doing it all in my free time. But before mr. Putin will close Internet in Russia I’d like to demonstrate that it’s possible to write very fast binary deserializer taking in account several assumptions which seems quite reasonably for me. So, feel free to borrow the ideas, or share the ideas, if you know how it could be implemented better. -- Sincerely, Stanislav Chernichkin.

4 4

Re: [Haskell-cafe] Yet another binary reader (10x faster than Hackage's binary)
by Olaf Klinke 21 Mar '19

21 Mar '19

My (admittedly limited) experience with parsers, of which deserializers are a special case, is that with complicated data structures the majority of computing time is spent transforming the raw data (i.e. chunks of bytes) into meaningful information such as Doubles, Timestamps, tree structures and so on. Of course you could argue that such conversions are not the scope of the deserializer and that these operations must be performed regardless which library one chooses. Nevertheless it somewhat relativizes the importance of the choice of serializing library. Your benchmarks do not go beyond adding integers as far as I can see, which is a relatively cheap operation. I'd be interested in a benchmark where e.g. a huge Set or an array of custom records are used. You seem to work in bioinformatics. There are plenty of examples from this field, e.g. genomic annotations. Cheers, Olaf

1 0

Re： Yet another binary reader (10x faster thanHackage's binary; )
by Dr.Koster 20 Mar '19

20 Mar '19

These two assumptions are basically the most challenging issues stop people from getting more performance, e.g. If somehow you want to support incremental parsing, a primitive like `ensureN` probably need to do something like ``` ensureN n = Parser $ \ buf ... k -> if n < length buf then k buf ... else partial buf ... k ``` The k appear twice inside `ensureN` 's both branch, which will stop parsers from getting inline sooner or later since inlining k into both branch will produce exponential amount of code, so instead GHC will allocate k and make a jump, the best we can get is probably to try make it a joint point, so no allocation is needed, but that's the best you can get from argument passing buffers. Regards Han Dong ------------------ 原始邮件 ------------------ 发件人: "Joachim Durchholz"<jo(a)durchholz.org>; 发送时间: 2019年3月20日(星期三) 中午1:28 收件人: "haskell-cafe"<haskell-cafe(a)haskell.org>; 主题: Re: [Haskell-cafe] Yet another binary reader (10x faster thanHackage's binary; ) Am 19.03.19 um 22:58 schrieb Станислав Черничкин: > possibly because of inlining of > specialization magic. I have seen people validate this kind of assumption, by looking at the various intermediate representations of code. If you're after performance, you may want to do that. You may find that things work differently than expected, with the potential for becoming a bottleneck when stuff is scaled up (more varied datatypes in the input stream, or more complex target datatypes, stuff hidden behind "oh I don't need GC yet", that kind of finding). Regards, Jo _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

2 1

Data.Functor.Compose Combinator
by Ben Kolera 20 Mar '19

20 Mar '19

Hi, I was wondering if this function exists somewhere (i'm not attached to the name): (<<$>>) :: (Functor f) => (a -> g b) -> f a -> Compose f g b (<<$>>) mkG = Compose . fmap mkG It's handy if you're working with an applicative (e.g Reflex.Dynamic) and want to layer on top a validation type function that returns a different applicative that need composing. So I figure that it must exist somewhere! :) E.g. data Person = Person Text Email nameTextDyn :: Dynamic t Text emailTextDyn :: Dynamic t Text validEmail :: Text -> Validation (NonEmpty Text) Email validPersonDyn :: Dynamic t (Validation (NonEmpty Text) Person = getCompose $ Person <$> (pure <<$>> name) <*> (validEmail <<$>> emailTextDyn) Cheers, Ben

1 0

Fwd: [Fwd: linux version compatible with GHC 4.3 ?]
by Damien Mattei 19 Mar '19

19 Mar '19

---------- Forwarded message --------- From: Damien Mattei <damien.mattei(a)univ-cotedazur.fr> Date: Mon, Mar 18, 2019 at 11:46 AM Subject: [Fwd: linux version compatible with GHC 4.3 ?] To: <damien.mattei(a)gmail.com> ---------- Forwarded message ---------- From: Damien Mattei <damien.mattei(a)univ-cotedazur.fr> To: Haskell Cafe <haskell-cafe(a)haskell.org> Cc: Bcc: Date: Mon, 18 Mar 2019 11:39:25 +0100 Subject: linux version compatible with GHC 4.3 ? Hi, does anyone know a linux distrib where haskell platform installs out of the box? it fails on fedora core 28 (32bits) : [root@localhost Downloads]# ghci /usr/local/haskell/ghc-8.4.3-i386/lib/ghc-8.4.3/bin/ghc: error while loading shared libraries: libtinfo.so.5: cannot open shared object file: No such file or directory but works on older system such as CentOS Linux release 7.2.1511 (Core) 64bits: [mattei@moita ~]$ cat /etc/redhat-release CentOS Linux release 7.2.1511 (Core) [mattei@moita ~]$ ghci GHCi, version 8.4.3: http://www.haskell.org/ghc/ :? for help Prelude> does it works on Ubuntu? Damien

4 3

Hard cases for "Note 5" in Haskell layout parsing
by Chris Smith 18 Mar '19

18 Mar '19

Quick question. I'm trying to resolve Haskell layout in order to provide better syntax highlighting, but without committing to building a full Haskell parser. The reason this is hard at face value is because of the "Note 5" in the relevant part of the Haskell Report, which says that a layout context closes whenever (a) the next token without closing the layout context would not be the start of any valid Haskell syntax, but (b) the current statement followed by an implicit '}' could be valid Haskell syntax. This rule is why it's okay to write "let x = 5 in x^2", even though let introduces a new layout syntax: the "in" implicitly closes it because "x = 5 in" isn't the start of any valid syntax, so the layout context is implicitly closed before the "in". Now for my question. Does anyone know other cases besides let/in where this commonly comes up? Everywhere I've seen this before uses let/in as an example, but then concludes that full parsing is needed and gives up on simpler answers. But the specific example with let/in is easily handled with a special-purpose rule that closes layout contexts as needed when an "in" keyword shows up. I cannot seem to construct any other examples, because other layout-introducing keywords have their contents at the end of syntactic element. Is there something I've missed here? I don't even care if it's a situation where something is technically valid but a horrible edge case. I'm interested in realistic counterexamples. Thanks, Chris

3 3

Learn you a Haskell for great good! Group readnig
by Jaysinh shukla 17 Mar '19

17 Mar '19

Respected Member, I am trying to learn Haskell from the book Learn you a Haskell for great good! [1]. I livestream my reading sessions at this [2] Twitch channel. Group reading is helping me to learn better. It is way more productive than reading alone. If you are learning Haskell, please join us. If you already know Haskell, please join to teach. Teaching will give you a good revision. Don't worry about the commitment to attend each and every session. It is fine to attend sessions on the day you are available and skip the rest. The fun part is, all the sessions will be on weekends. You can watch videos of past sessions here [3]. I am trying to maintain an event calendar for upcoming sessions here [4]. Please share these resources with whoever learning the Haskell. Don't hesitate to ask your questions here or in a personal mail. Thanks! 1: http://learnyouahaskell.com 2: https://www.twitch.tv/jaysinhp 3: https://www.twitch.tv/jaysinhp/videos 4: https://www.twitch.tv/jaysinhp/events

1 0

Preferred location for symlink-bindir?
by David Banas 16 Mar '19

16 Mar '19

Hi all, In moving to the cabal 'new-install' command, is there a preferred definition for the ‘symlink-bindir’ cabal configuration variable on Mac OS X? Thanks, -db

1 0

initialization of AlexUserState
by Kees Bleijenberg 15 Mar '19

15 Mar '19

Hi All, I have a Alex lexer with wrapper "monadUserState-bytestring". The AlexUserState = AlexUserState {sheetNames:: {String], currSheet:: Int}. The lexer should check whether a found sheetname is in sheetNames. The sheetNames from AlexUserState come from outside. I want to invoke the lexer many times with different AlexUserStates. Therefore the alexInitUserState function is not what I want. I'de like to initialize the AlexUserState while starting the lexer, i.e. as a parameter to runAlex. Can that be done? Kees --- Dit e-mailbericht is gecontroleerd op virussen met Avast antivirussoftware. https://www.avast.com/antivirus

2 1