At 16:53 22/06/04 +0300, Max Ischenko wrote:

Hi all,

I'm going to try to implement a version of Markdown tool[1] in Haskell. The application is rather simple: take a text file with some (simple) mark-up embedded in it and turn it into another text file, this time with XHTML markup.

Cool project!

I need some guidelines on how to get started. I'll have to struggle with both the language itself (as I am a newcomer) plus with libraries that I may need.

Ah, how to get started? I find myself wrestling with this each time I start a new Haskell project, or phase of a project.

If I to do this in some convenient language, like Python, I'd use a lexer to parse the input, a state machine to build some internal representation and some serializer to write this into XHTML.

These choices are all available in Haskell. The way you pit them together may be a little different (see below). It's bee a while since I peered at MarkDown, and I forget how the language looks, except that I recall that, by design, it's kept very simple and has very little "superfluous" syntactic structure. I think the first choice is whether to go for a separately identifiable lexing phase, rather than working directly from the raw text. Either might work, I think. The HaXml XML parser has a separate lexer, but it turns out that it's not always easy to get the tokenization right without having contextual information (e.g. from the syntax analyzer). (XML is rather messy in that way.) In Haskell, it's often reasonably efficient to construct a program as a composition of "filters", rather like a Unix command pipeline; lazy evaluation often means that data can "stream" through the filters and never exist in its entirety in an intermediate form. This immediately allows the program structure to be resolved into a number of smaller, independent pieces; e.g. tokenize :: String -> [Token] parse :: [Token] -> DocModel createXHTML :: DocModel -> Document -- (cf. HaXml) Then HaXml provides function that can generate textual XML. Thus the overall conversion function might look like: markdownToXHTML :: String -> String markdownToXHTML = show . document . createXHTML . parse . tokenize (where "document" is from the HaXml module Text.XML.HaXml.Pretty). For parsing of any complexity, I recommend Parsec: it has the advantage of being very well documented, and it helps to show how monads can be used to handle state information. The outline sketched above has at least one weakness, it doesn't provide any way to handle errors. This could be overcome by using Either as an error monad (see Control.Monad and Control.Monad.Error in the standard hierarchical libraries), and then using >>= in place of function composition (noting the reversal of component order): -- Get definition of (Either String) as an instance of Monad: import Control.Monad() import Control.Monad.Error() tokenize :: String -> Either String [Token] parse :: [Token] -> Either String DocModel createXHTML :: DocModel -> Either String Document markdownToXHTML :: String -> Either String String markdownToXHTML s = tokenize s >>= parse >>= createXHTML >>= (Right . show . document) Or, using the do-notation: markdownToXHTML s = do { ts <- tokenize s ; dm <- parse ts ; xd <- createXHTML ; return $ show (document xd) }

Which approaches could you suggest for that kind of application in Haskell? I'd especially glad to hear about some "idiomatic" or "native" ways to solve the problem.

I've made a quick search and found tools like Parsec, HaXML, Happy and WASH/HTML. Not sure which ones I'll need.

I recommend Parsec (if it fits your needs, of course). It's not clear to me whether you want to generate text directly from the document data, or via something like HaXml. HaXml does have a pretty printing function (though I have noticed some minor problems with it, though probably nothing you'd encounter with this application.) ... Just to check out my use of >>= and do-notation, I constructed a trivial complete program using both. [[ -- spike-composition.hs -- -- composition of functions; using Either as an error monad -- Get definition of (Either String) as an instance of Monad: import Control.Monad() import Control.Monad.Error() data Token = T { unT::Char } data DocModel = D String instance Show DocModel where show (D s) = s tokenize :: String -> Either String [Token] tokenize = Right . map T parse :: [Token] -> Either String DocModel parse = Right . D . reverse . map unT docToString1 s = tokenize s >>= parse >>= (Right . ("docToString1: "++) . show) docToString2 s = do { ts <- tokenize s ; dm <- parse ts ; return $ (("docToString2: "++) . show) dm } t1 = docToString1 "abc" -- == Right "docToString1: cba" t2 = docToString2 "def" -- == Right "docToString2: fed" ]] #g --

[1] - http://daringfireball.net/projects/markdown/

TIA, max. _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

------------ Graham Klyne For email: http://www.ninebynine.org/#Contact