- Haskell-Cafe - Haskell.org

Re: [Haskell-cafe] Alternative instance for non-backtracking parsers
by Olaf Klinke 03 Sep '18

03 Sep '18

> Hello, Olaf. I have some distrust of elegant solutions (one of them are > C.P. libs). I have a program that parses several CSV files, one of them 50MB in size, and writes its result as HTML. When I started optimizing, the execution time was 144 seconds. Profiling (thanks to Jasper Van der Jeugt for writing profiteur!) revealed that most of the time was spent parsing and postprocessing the 50MB CSV file. Changing the data structure of the postprocessing stage cut down the execution time to 32 seconds, but still the majority is spent on parsing. Then I realized that (StateT String Maybe) is a parser which conveniently has all the class instances one needs, most notably its Alternative instance make it a backtracking parser. After defining a few combinators I was able to swap out my megaparsec parser against the new parser, which slashed execution time in half. Now most of the parsing time is dedicated to transforming text to numbers and dates. I doubt that parsing time can be reduced much further [*]. The new parser was identical to the old parser, only the combinators now come from another module. That is the elegant thing about monadic parser libraries. I will now use the fast parser by default, and if it returns a Nothing, the program will suggest a command line flag that switches to the original megaparsec parser, exactly telling the user where the parse failed and why. I am not sure whether there is another family of parsers that have interfaces so similar that switching from one package to another is as effortless as monadic parsers. Cheers Olaf [*] To the parser experts on this list: How much time should a parser take that processes a 50MB, 130000-line text file, extracting 5 values (String, UTCTime, Int, Double) from each line?

7 8

Testing of GHC extensions & optimizations
by Rodrigo Stevaux 03 Sep '18

03 Sep '18

Hi, For those familiar with GHC source code & internals, how are extensions & optimizations tested? And what are the quality policies for accepting new code into GHC? I am interested in testing compilers in general using random testing. Is it used on GHC?

5 8

Applying typed hole proposal leads to compilation failure
by Vlatko Basic 03 Sep '18

03 Sep '18

Hello Cafe, I encountered a strange issue regarding typed hole. If I ask GHC (8.4.3) for a typed hole proposal, it says 'use c', but if I use 'c', it complains that it can't unify 'c' and 'c1'. Why does GHC think those two are different and how to tell to GHC they are the same? Best regards, vlatko *Typed hole proposal:* • Found type wildcard ‘_c’ standing for ‘c’ Where: ‘c’ is a rigid type variable bound by the type signature for: f1 :: forall (m :: * -> *) c. MonadIO m => c -> m Bool at Test.hs:15:1-32 • In the type signature: run :: MS _c Int a -> (Either String a, Int) *Error when typed hole used:* • Couldn't match type ‘c1’ with ‘c’ ‘c1’ is a rigid type variable bound by the type signature for: run :: forall c1 a. MS c1 Int a -> (Either String a, Int) at Test.hs:21:5-47 ‘c’ is a rigid type variable bound by the type signature for: f1 :: forall (m :: * -> *) c. MonadIO m => c -> m Bool at Test.hs:15:1-32 Expected type: MS c1 Int a -> (Either String a, Int) Actual type: MS c Int a -> (Either String a, Int) * * *Minimal runnable example:* {-# LANGUAGE ScopedTypeVariables, PartialTypeSignatures #-} module Test where import Prelude import Control.Monad.IO.Class (MonadIO) import Control.Monad.Trans.Except (ExceptT, runExceptT) import Control.Monad.Trans.Reader (Reader, runReader) import Control.Monad.Trans.State (StateT, runStateT) type MS c s = ExceptT String (StateT s (Reader c)) runMS :: c -> s -> MS c s a -> (Either String a, s) runMS c s f = runReader (runStateT (runExceptT f) s) c f1 :: (MonadIO m) => c -> m () f1 c = do let _x1 = run f2 let _x2 = run f3 return () where run :: MS *_c*Int a -> (Either String a, Int) --- failure or success, 'c' or '_' run = runMS c 0 f2 :: MS c s Bool f2 = pure False f3 :: MS c s [Int] f3 = pure []

3 4

Re: [Haskell-cafe] Alternative instance for non-backtracking parsers
by Olaf Klinke 02 Sep '18

02 Sep '18

Thanks Bardur for the pointer to bytesting-lexing and cassava. The problem is that my csv files are a little bit idiosyncratic. That is, they have BOM, semicolons as separators, The fractional numbers are in the format 1.234,567 and there are dates to parse, too. I tried parseTimeM from the time package, but that is slower than my own parser. That said, my megaparsec parser seems to spend quite some time skipping over text with regex ';"[^"]*"', that is, fields whose content does not concern the application at hand. Hence using a CSV library for tokenizing might be a good idea. Thanks to Peter Simons for pointing out that cassava indeed has attoparsec as a dependency. Maybe CSV is a red herring, after all. It's just a some text-based syntax where the count of semicolons indicate where in the line I can find the data I'm interested in. I'm more curious about how to make number conversion fast. I've looked at the source of megaparsec-6.5.0, attoparsec-0.13.2.2, bytestring-lexing-0.5.0.2 and base-4.11.1.0. They all do the same thing: Convert digits to numbers individually, then fold the list of digits as follows: f x digit = x * 10 + value digit number = foldl' f 0 digits For 'value' above, Megaparsec uses Data.Char.digitToInt while Attoparsec uses Data.Char.ord. I also rolled my own Double parser for locale reasons. Are there any libraries that handle all the formats 1,234.567 1234.567 1.234,567 1234,567 maybe by specifying a locale up front? It can't be done without, since 123.456 on its own is ambigous. Maybe the locale can be guessed from the context, maybe not, but that is certainly an expensive operation. MS Excel guesses eagerly, with occasional amusing consequences. Thanks to all who contributed to this thread so far! Olaf

1 0

Re: [Haskell-cafe] if-then without -else?
by Rodrigo Stevaux 31 Aug '18

31 Aug '18

> > which cannot be prohibited statically > if we want both the language to be Turing complete, > and type inference to be decidable. > > Could you be so kind as to give a hint on literature that could help me understand this statement?

2 2

Re: [Haskell-cafe] Alternative instance for non-backtracking parsers
by Olaf Klinke 31 Aug '18

31 Aug '18

>Hello, Olaf. I have some distrust of elegant solutions (one of them are >C.P. libs). I am a mathematician, I trust a lot in elegance and beauty. As long as the performance penalty is not prohibitively large, I'd prefer a beautiful implementation over an aggressively optimized one. At least when the code does not live in some real-time system. Certainly there are cases where you absolutely need speed or low memory footprint. > For example, I read that they have problems with memory >consuming (famous article about arrow-based parsers), also your example >(my own experience includes a lot of tricks which I don't like)... So >(but this is my IMHO) for big and serious project with parsing of >something complex I will use bindings to good known parser libraries... I consider Parsec and it descendants good and known. I could not write something like that. Currently I rely on megaparsec because of its informative error messages, knowing that I can switch to one of the siblings without much effort when I need more speed. >Maybe something like this: https://github.com/markwright/antlrc. Btw, >Happy looks still live (https://github.com/simonmar/happy) Have you used any of the above? > >I use "Read"-style parsers for small constructions only, so fail of such >small parser is only case where error is producing and I do it >explicitly (as biggest example: header of Git patch). > >Olaf, what do you think about > 1. http://hackage.haskell.org/package/frisby Frisby looks quite old, as it has idiosyncratic combinators for stuff now in Data.Functor. I've never used it, so I can't tell what its strengths are. Can you? > 2. http://tanakh.github.io/Peggy Notice that in the examples of bothy frisby and peggy, parsing a number only parses a string of digits, then passes it to 'read', i.e. another parser. Is that efficient, elegant parsing? I'm nevertheless intrigued by PEG parsers, because I once read about the connection between grammars and compression: A set of production rules can be quite a bit smaller than the actual text. This turns parsing upside down: The text is constant, the grammar unknown. Olaf

2 1

Re: [Haskell-cafe] Haskell-Cafe Digest, Vol 180, Issue 32
by Michal J Gajda 31 Aug '18

31 Aug '18

I got 4.7s for similar amount of data in 2013. However I was pretty sure that fully inlined implementation could potentially go 5x faster. http://hackage.haskell.org/package/hPDB Please check xeno XML parser benchmarks for another example. https://hackage.haskell.org/package/xeno On Fri, 31 Aug 2018 at 14:41, <haskell-cafe-request(a)haskell.org> wrote: > Send Haskell-Cafe mailing list submissions to > haskell-cafe(a)haskell.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe > or, via email, send a message with subject or body 'help' to > haskell-cafe-request(a)haskell.org > > You can reach the person managing the list at > haskell-cafe-owner(a)haskell.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Haskell-Cafe digest..." > Today's Topics: > > 1. Re: HDBC packages looking for maintainer (Tobias Dammers) > 2. Re: Alternative instance for non-backtracking parsers > (Olaf Klinke) > 3. Re: Alternative instance for non-backtracking parsers > (Bardur Arantsson) > > > > ---------- Forwarded message ---------- > From: Tobias Dammers <tdammers(a)gmail.com> > To: haskell-cafe(a)haskell.org > Cc: > Bcc: > Date: Thu, 30 Aug 2018 15:24:04 +0200 > Subject: Re: [Haskell-cafe] HDBC packages looking for maintainer > Hi, > > I'd be interested. I've used HDBC on a few projects, and my yeshql > library was originally built with HDBC as the only backend. It would be > a terrible shame to see this bitrot. > > Cheers, > > Tobias (tdammers on github etc.) > > On Mon, Aug 13, 2018 at 12:07:38PM +0200, Erik Hesselink wrote: > > Hi all, > > > > I've been the maintainer for some of the HDBC packages for a while now. > > Sadly, I've mostly neglected them due to lack of time and usage. While > the > > packages mostly work, there are occasional pull requests and updates for > > new compiler versions. > > > > Because of this I'm looking for someone who wants to take over HDBC and > > related packages [1]. If you use HDBC and would like to take over > > maintainership, please let me know and we can get things set up. > > > > Regards, > > > > Erik > > > > [1] https://github.com/hdbc > > > _______________________________________________ > > Haskell-Cafe mailing list > > To (un)subscribe, modify options or view archives go to: > > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe > > Only members subscribed via the mailman list are allowed to post. > > > -- > Tobias Dammers - tdammers(a)gmail.com > > > > > ---------- Forwarded message ---------- > From: Olaf Klinke <olf(a)aatal-apotheke.de> > To: PY <aquagnu(a)gmail.com> > Cc: haskell-cafe <haskell-cafe(a)haskell.org> > Bcc: > Date: Thu, 30 Aug 2018 20:21:07 +0200 > Subject: Re: [Haskell-cafe] Alternative instance for non-backtracking > parsers > > Hello, Olaf. I have some distrust of elegant solutions (one of them are > > C.P. libs). > > I have a program that parses several CSV files, one of them 50MB in size, > and writes its result as HTML. When I started optimizing, the execution > time was 144 seconds. Profiling (thanks to Jasper Van der Jeugt for writing > profiteur!) revealed that most of the time was spent parsing and > postprocessing the 50MB CSV file. Changing the data structure of the > postprocessing stage cut down the execution time to 32 seconds, but still > the majority is spent on parsing. > Then I realized that (StateT String Maybe) is a parser which conveniently > has all the class instances one needs, most notably its Alternative > instance make it a backtracking parser. After defining a few combinators I > was able to swap out my megaparsec parser against the new parser, which > slashed execution time in half. Now most of the parsing time is dedicated > to transforming text to numbers and dates. I doubt that parsing time can be > reduced much further [*]. The new parser was identical to the old parser, > only the combinators now come from another module. That is the elegant > thing about monadic parser libraries. > I will now use the fast parser by default, and if it returns a Nothing, > the program will suggest a command line flag that switches to the original > megaparsec parser, exactly telling the user where the parse failed and why. > I am not sure whether there is another family of parsers that have > interfaces so similar that switching from one package to another is as > effortless as monadic parsers. > > Cheers > Olaf > > [*] To the parser experts on this list: How much time should a parser take > that processes a 50MB, 130000-line text file, extracting 5 values (String, > UTCTime, Int, Double) from each line? > > > > ---------- Forwarded message ---------- > From: Bardur Arantsson <spam(a)scientician.net> > To: haskell-cafe(a)haskell.org > Cc: > Bcc: > Date: Thu, 30 Aug 2018 21:43:55 +0200 > Subject: Re: [Haskell-cafe] Alternative instance for non-backtracking > parsers > On 30/08/2018 20.21, Olaf Klinke wrote: > >> Hello, Olaf. I have some distrust of elegant solutions (one of them are > >> C.P. libs). > > > > [*] To the parser experts on this list: How much time should a parser > take that processes a 50MB, 130000-line text file, extracting 5 values > (String, UTCTime, Int, Double) from each line? > > Not an expert, but for something as (relatively!) standard as CSV, I'd > probably go for a specialized solution like 'cassava', which seems like > it does quite well according to https://github.com/haskell-perf/csv > > Based purely the lines/second numbers on that page and the number you've > given, I'd guesstimate that your parsing could potentially be as fast as > (3.185ms / 1000 lines) * 130000 lines = 414.05ms = 0.4 s. > > (Of coure that still doesn't account for extracting the Int, Double, > etc., but there are also specialized solutions for that which should be > pretty hard to beat, see e.g. bytestring-lexing.) > > It's also probably a bit less elegant than a generic parsec-like thing, > but that's to be expected for a more special-case solution. > > Regards, > > > _______________________________________________ > Haskell-Cafe mailing list > Haskell-Cafe(a)haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe >

2 1

Installing hsdev on windows
by Yotam Ohad 31 Aug '18

31 Aug '18

Hi cafe, I installed haskell platform for windows 10 and then ran `cabal install hsdev` I got the following error: Resolving dependencies... cabal: Could not resolve dependencies: [__0] trying: hsdev-0.3.1.4 (user goal) [__1] trying: hlint-2.1.10 (dependency of hsdev) [__2] trying: haskell-src-exts-1.20.2 (dependency of hsdev) [__3] next goal: ghc-syb-utils (dependency of hsdev) [__3] rejecting: ghc-syb-utils-0.3.0.0 (conflict: hsdev => ghc-syb-utils>=0.2.3 && <0.3) [__3] trying: ghc-syb-utils-0.2.3.3 [__4] next goal: ghc (dependency of hsdev) [__4] rejecting: ghc-8.4.3/installed-8.4..., ghc-8.4.3, ghc-8.4.1 (conflict: ghc-syb-utils => ghc>=7.0 && <8.4) [__4] rejecting: ghc-8.2.2, ghc-8.2.1 (conflict: hsdev => ghc==8.4.*) After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: hsdev, haskell-src-exts, ghc-mod, hlint, base, ghc, ghc-syb-utils Any Ideas on how to solve this? I'm clueless thanks

3 2

Parsing Haskell with Layout
by Benjamin Redelings 30 Aug '18

30 Aug '18

Hi, Section 10.3 of the Haskell 2010 report describes the layout rule for Haskell in terms of rules for inserting curly braces and semicolons into the token stream. I'm not sure of the theory behind implementing Note 5 in section 10.3. This says roughly that we insert a "}" token after token t[n] if the tokens t[1]...t[n+1] have no valid parses, and the sequence of tokens t[1] ... t[n] followed by "}" DO have a valid parse. I think this allows you to write f z = let x=2;y=3 in x+y+z However, except for Note 5, it seems like you could do a simple preprocessing step on the token stream to insert "{", ";", and "}" before parsing. Note 5 seems more complicated. (I feel like I read a note somewhere that said "read this and weep" and then failed to implement Note 5. But now I can't remember where I read this.) In simple explanations like Write you a haskell (http://dev.stephendiehl.com/fun/008_extended_parser.html) it seems that the approach is simply to ignore Note 5. Can anyone fill me in on (a) what concepts are assumed by the report that would make it "obvious" how Note 5 is supposed to be implemented? (b) a simple (hopefully) implementation strategy for implementing it? I'm looking for a language-independent approach for parsing. I did see some mention of error-correcting parsers in the Parsec paper, which might allow adding the "}" at a parse error. But I see there is a paper called " An error correcting parser for context free grammars that takes less than cubic time". The "less than cubic time" seems rather worrisome, so I don't know if this is the right approach. Also, when using parsec, it seems that you might get an idea how many tokens are a valid prefix of the grammer when the token stream fails to parse. Hence, you could try inserting a "}" into the token stream at the point of a parse error. However, a naive approach would then require re-parsing the whole stream from the beginning until each parse error, which seems very expensive. Thanks for any help! -BenRI

3 3

Dependent types, weak references, and stable names
by David Feuer 29 Aug '18

29 Aug '18

I just realized that dependent types should let us do some things with weak references and stable names less awkwardly. In particular, there are two invariants that are not (and cannot be) expressed in the type system at present: 1. If two stable names are equal, then their underlying objects are equal. 2. If a weak reference was built using a key and a value, and the key is alive, then the value is also alive. I hope we'll be able to express these with Dependent Haskell, using whatever the real syntax will be. data Weak :: k -> Type -> Type mkWeak :: (key :: k) -> v -> IO () -> IO (Weak key v) deRefWeakSurely :: (key :: k) -> Weak key v -> IO v -- Intentionally omitted: finalize data StableName :: k -> Type makeStableName :: (key :: k) -> IO (StableName key) eqStableName :: StableName x -> StableName y -> Maybe (x :~: y) The idea is that we can calculate the StableName of a key, look for that that in a hash table of weak pointers, and if we find it, we get evidence that we can use to dereference the weak pointer (deRefWeakSurely would use touch# under the hood, but that's not our concern).

2 1