-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Greetings, In light of fairly recent discussion, on this mailing list, I decided to investigate the topic of Markdown support for Haddock. The archives of the recent discussion can be seen at [1]. This post aims to summarise the current state of discussion. I do aim to file a proposal for a GSoC project on this issue but it'd be foolish to file a proposal for a project aiming to benefit the community without consulting the community itself. Here are some main points and ideas gathered: * reSt seems to have a small following - quite a bit smaller than the Markdown community. In fact, there seems to be a significant amount of people pushing for Markdown which contrasts what we can read in a topic from 2008 at [2]. I guess it just shows how much Markdown has gained in popularity in recent years. * There are issues with using Markdown even before we attempt to use it for Haskell documentation: * There exists no formal specification or semantics. It would seem that a significant number of Markdown parsers are creating by reverse engineering an already existing parser. This is bad because we end up propagating the bugs and workarounds around ambiguity that the original parser has. * As a follow-up to the previous point, the (vanilla) Markdown is ambiguous and there is nothing to resolve it. As Richard A. O'Keefe pointed out, there exist situations where it's not possible to infer the semantics of Markdown from its official implementation and the result is parser/writer-specific [6]. * John MacFarlane has already written a Markdown parser in Haskell. It can be read at [3]. This means that the new extension would not need to rely on Pandoc. He says ``I have an experimental thing here that could be used as a basis (it's 7x faster than pandoc and uses 1/5 the memory, BSD licensed)''. This is great! The post can be seen in full at [4]. * An alternative idea was to simply write a writer module for Pandoc for Haddock. * A reader module is already present. * According to John MacFarlane, Haddock is not expressive enough to take advantage of this. Furthermore, Pandoc doesn't have some constructions that Haddock does [4]. * Comes back to the problem on relying on such a large package as Pandoc. * Yet another proposal was rather than introducing Markdown to concentrate on fixing current issues and adding features to Haddock as it stands [8]. This is one of the options listed at the short blog post at [14] by Johan Tibell. * David Waern, one of Haddock maintainers admits that Haddock lacks active development and it has been that way for a longer time. Having said that, he seems to believe that Markdown integration is a project that can realistically be completed over summer. Such project would be able to use the existing HTML backend in Haddock. [9]. * Math expressions were requested and MathJAX was suggested as a solution at [5]. math.stackexchange.com uses MathJAX and it works quite well. Personally, I believe that Haskell documentation would benefit from this simply due to the academic nature of the language. * Support for Literate Haskell would be a welcome addition as suggested by Andrew Butterfield at [7]. * There are issues with CPP and LHS in regards to using Markdown in documentation. They are pointed out at [10] by John MacFarlane and others that he's replying to. * As pointed out 5 years ago at [11], we'd have to do some preprocessing on current, fairly critical sections of comments used by Haddock. I believe these are fairly useful and it would be a shame to see them go. I hope I haven't missed anything of high importance in a list above. When researching the topic, issues with Markdown quickly become apparent. Today, there are tens of different Markdown flavours: each company has different needs and each company interprets the vague original documentation in a way that's convenient to them. In these situations, topics, e-mails and blog posts like this one happen. There is a call to action from October 2012 at [12] but there seems to be absolutely no progress on any of the miraculous things mentioned in the post. As a result of that post, a W3C community was formed at [13]. It's clear that the community is inactive and no progress towards a solution was made. Having considered all information gathered here, I believe this would make a good GSoC project. There has been interest in this for Haskell since 2008 judging by the size of the last thread, it's clear that the community interest is high. In fact, it surprises me that this hasn't been done in previous years. The main issue that I see with using Markdown is the fact that we can't simply add a {-# HADDOCK Markdown #-} and let some parser rip through it. There are many issues that need to be resolved that I outlined above. This will no doubt result in a Markdown flavour suited to documenting Haskell rather than the wonderful, standardised Markdown that [13] is talking about. Is this different in any way from what GitHub or StackOverflow is doing? Introducing yet another flavour? In essence, no. For all Haskellers however, there would be a difference: this is a chance to use the pleasant-to-eye Markdown that many seem to like and use it for documentation. We have the chance to formalise it and make it work properly. It's not a perfect solution for every single person writing documentation out there but it's a good solution for Haskellers. Obviously, the ideal would be to have the wonderful, standardised, formalised Markdown that [13] seems to try and conjure and then extend that. Alas, we have no such luxury. As a closing paragraph, I would like to ask some questions. - - Is creating yet another Markdown flavour acceptable? I see no better solution: we'll have to take Markdown and fix it up to our needs. I'm sure most people are aware of the fairly famous Internet web-comic relating to standards [15]. We're however not trying to push a new standard - writing a proper standard, implementing it and then extending it for Haskell documentation is far too much for a single summer. - - Is there a mentor that would be willing to oversee the project? There's a fairly fresh ticket at [16] but there seems to be no indication of mentor-ship. I'm very interested in the project - it seems to hold high value for the community while not being completely out of scope for a student with a couple of months of Haskell experience, especially considering the large ``Getting up to scratch'' period. Thank you for your time. Links - ----- [1] - http://comments.gmane.org/gmane.comp.lang.haskell.cafe/104398 [2] - http://www.haskell.org/pipermail/haskell-cafe/2008-February/039846.html [3] - https://github.com/jgm/Markdown [4] - http://permalink.gmane.org/gmane.comp.lang.haskell.cafe/104421 [5] - http://permalink.gmane.org/gmane.comp.lang.haskell.cafe/104443 [6] - http://permalink.gmane.org/gmane.comp.lang.haskell.cafe/104422 [7] - http://permalink.gmane.org/gmane.comp.lang.haskell.cafe/104428 [8] - http://permalink.gmane.org/gmane.comp.lang.haskell.cafe/104523 [9] - http://permalink.gmane.org/gmane.comp.lang.haskell.cafe/104545 [10] - http://permalink.gmane.org/gmane.comp.lang.haskell.cafe/104444 [11] - http://www.haskell.org/pipermail/haskell-cafe/2008-February/039939.html [12] - http://www.codinghorror.com/blog/2012/10/the-future-of-markdown.html [13] - http://www.w3.org/community/markdown/ [14] - http://blog.johantibell.com/2013/04/haskellorg-gsoc-ideas.html [15] - http://xkcd.com/927/ [16] - http://trac.haskell.org/haddock/ticket/244 - -- Mateusz K. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (GNU/Linux) iQIcBAEBAgAGBQJRdt+WAAoJEM1mucMq2pqXspkP/j+SCyfho65/ji5OvNCEBFTb YLGC1YY/dk/zbb67gDDxmFtxw/UBGnUifliGUKhCy2Zzm6mwF7BI1U4NWhbR2nkQ VWkysqHMn1JM8uQNtkdXEe/KUi2KxcM+VtOkuCUQeF0mlHB7Hthlz58lGZHD+cMK EvhN0BGrL1/xnKwSexTSNexXIDCmvCW19jTxuR9uwPqVNVPMjWxO0wRVplnzcqyf S8jeG0+Q28S0Su/CxNUdUQ+pE2+FWksNY4a/d/8Tce9sFoTTXUEGAp6Nbgyi+uKd yS+AtBqvcsw7KYw03kQAB1qCD05dtc7ptp9ohjgwYY6jzgRMBrIRdDUvmqT6nPh1 MHgG9rkg9yj+gKhSlNiTZ7MCl/o/CkCtPG0rNtd6//QXDZlH3Xx6CpYKm/GnyAWh a2vX5ZX0urVIapFPPEpClMkleLbHfi2UWGVbC43jP0lvsZTMlr4uvwHzJ8N4e9AI 2MdK4yZes6b/DSH47FPdoAeYksUvwI0xpI5rpkLuoOh+O094gQVhm4yb+ceqF6CE VddqnHPPF4gn7NTeYh6fnsAvzCm5PYgyhEUySbkmlGHjAODtvs6uC9MW4zwYPeCp BvGTSsGx7DbtjHwDYQIzMI/oyjFXMVsQ/+bE+pzIo/tQJevXkILaZKYhBvUyAbE5 ypdJkLdQb8EcWGT5Cx0v =sTE0 -----END PGP SIGNATURE-----