Am 31.10.2010 16:50, schrieb Ozgur Akgun:

I don't know if you've already used it, but Parsec includes some kind of a lexer through the Language

http://hackage.haskell.org/packages/archive/parsec/3.1.0/doc/html/Text-Parse...

and Token

http://hackage.haskell.org/packages/archive/parsec/3.1.0/doc/html/Text-Parse...

modules. You can start by having a look at the makeTokenParser

http://hackage.haskell.org/packages/archive/parsec/3.1.0/doc/html/Text-Parse... function. Yeah, I've read about the TokenParser, but since my language is not a typical programming language it's use is very limited for me. My language is basicly a combination of a scripting language and a markup language like, for example, markdown. And parsing that scripting language isn't the difficult part so far...

Am 31.10.2010 16:53, schrieb Vo Minh Thu:

I can't really tell from your description, but maybe this is because of the way Parsec works when it deals with alternatives. When you combine several parsers with e.g. '<|>' or 'choice', an alternative that can consume some input but fails will make the whole combined parser fail too. So you have to either factorize you parsers or use the 'try'. See the documentation for 'try' at http://hackage.haskell.org/packages/archive/parsec/3.1.0/doc/html/Text-Parse... .

This is exactly what gives me headaches. It's hard to tell where you need try/lookAhead and where you don't need them. And I don't really feel comfortable wrapping everything into try blocks...

On 31 October 2010 16:23, Ozgur Akgun wrote:

Am I missing something?

Left factoring! :-)

Malcolm Wallace wrote:

Nils Schweinsberg wrote:

...

Vo Minh Thu wrote:

...

So you have to either factorize you parsers or use the 'try'.

This is exactly what gives me headaches. It's hard to tell where you need try/lookAhead and where you don't need them. And I don't really feel comfortable wrapping everything into try blocks...

Have you considered using a different set of parser combinators, a set that is actually composable, and does not require the mysterious "try"? I would recommend polyparse (because I wrote it), but uuparse would also be a fine choice.

I second that, the semantics of Parsec are quite tricky. It's worth learning them properly if you insist on using Parsec. If you don't want to do that, it's easier to use a different library. The two defining properties of Parsec's alternative <|> combinator are: 1) If a parser p consumes a character, then it will take precedence over its alternatives p <|> q = p . 2) Parsers that are in sequence with an alternative don't have any influence on which alternative is chosen. (p <|> q) >>= k = p >>= k if p succeeds q >>= k if p does not succeed but p >>= k might not succeed even if p does succeed and q >>= k would succeed. I found the following rule of thumbs helpful: * Try to use alternatives p <|> q only when p and q recognize different first characters. * Don't do recursion yourself, use premade combinators like many1 etc. instead. * lookAhead is very useful for avoiding capturing input in the second argument of manyTill Regards, Heinrich Apfelmus -- http://apfelmus.nfshost.com

On 31/10/2010 04:15 PM, Nils Schweinsberg wrote:

This is exactly what gives me headaches. It's hard to tell where you need try/lookAhead and where you don't need them. And I don't really feel comfortable wrapping everything into try blocks...

I vaguely recall somebody mentioning a parser library on Hackage where "try" is the default behaviour and you turn it off explicitly, rather than turning it on explicitly. Apparently this is much more intuitive. But unfortunately I can't remember what the hell the library was...

On Oct 31, 2010, at 17:15 , Nils Schweinsberg wrote:

Am 31.10.2010 16:53, schrieb Vo Minh Thu:

...

I can't really tell from your description, but maybe this is because of the way Parsec works when it deals with alternatives. When you combine several parsers with e.g. '<|>' or 'choice', an alternative that can consume some input but fails will make the whole combined parser fail too. So you have to either factorize you parsers or use the 'try'. See the documentation for 'try' at http://hackage.haskell.org/packages/archive/parsec/3.1.0/doc/html/Text-Parse... .

This is exactly what gives me headaches. It's hard to tell where you need try/lookAhead and where you don't need them. And I don't really feel comfortable wrapping everything into try blocks...

This is precisely why you should use a more general parser library like uu-parsing and try to avoid the more low-level techniques used in Parsec; uu-parsinglib avoids all the confusion arising from the use of try constructs. It furthermore gives you an online result, error correction and nice error messages. Its Utils module contains a lot of useful "standards" elements you might want to recognise. Thus far i have only happy users, and if you are having any problems please let me know. Doaitse PS: for all parsing libraries it holds that parsing times are negligeable when compared to the time spent on what you want to do with the parsed result.

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

The there is an amb combinator which you can use to lift an ambiguous parser into one which returns a list of results. This will take care that the part after the ambiguous non-terminal is only parsed once. Like most libraries I do indeed not cache results of applying parsers at specific positions, besides the use of amb. This is more than most other libraries do. Besides this there are various greedy versions of combinators which take care of repeating construct; so there is an pChainL and a pChainL_ng etc. This has a bit the same effect as the try combinator, but in a more structured way. Of course applying some left-factoring is always a good idea; but i think this should only be done once real problems arise, which is not very often. In case these solutions are not sufficient one could of course take one of the more advanced libraries, which performa real grammar analysis, such as a left corner transform. There are two libraries supporting this: the grammar-combinator package of Dominique deVriese and the Christmas tree package from Utrecht, which is mainly aimed at solving problems with the exponential complexity in GHC's implementation of the function read for values of data types with infix constructors. The first one uses many recent GHC extensions. Both are less of a combinator parsing style, since support for freely abstracting over common grammatical patterns is somewhat more problematic. Both can use uu-parsinglib as back end, Doaitse PS: previous implementations of GHC made some assumptions about the kind of programs people would write. Unfortunately some parsers using the older uulib package were thus spending 99.5% of their time in garbage collection. This has been cured, thanks to work by Simon Marlow. Some parsers run over 50 times faster!! On Jan 21, 2012, at 9:47 , Roman Cheplyaka wrote:

* S D Swierstra [2012-01-20 16:32:00+0100]

...

Thus far i have only happy users, and if you are having any problems please let me know.

Hi Doaitse,

One thing that I'm curious about is how you avoid combinatorial explosion when parsing in a breadth-first fashion. Do you somehow manage to merge the branches? Or just hope that in practice it won't be too bad?

-- Roman I. Cheplyaka :: http://ro-che.info/

On 31 October 2010 15:55, Stephen Tetley wrote: ecessary.

You can also write separate parsers this is covered in the (pdf) Parsec manual available from Daan Leijen's old home page, however I usually avoid this as it seems rather cumbersome.

D'oh. I meant "separate _scanners_" of course, see chapter 2.11 Advanced: Seperate scanners