Re: [Haskell-cafe] Hackage package "synopsis" sections

On 09/16/2014 05:13 AM, Michael Snoyman wrote:

On Tue, Sep 16, 2014 at 2:36 AM, Mateusz Kowalczyk wrote:

...

On 09/15/2014 09:11 PM, Michael Snoyman wrote:

...

I've had cases where some code I wanted to put in an example could not be expressed in haddock due to escaping rules.

The parser has been improved and the escaping rules have been made saner. You shouldn't have real problems since 2.14 release which came out along with GHC 7.8.1. If you have problems with escaping then I'd like to hear about it on the issue tracker rather than stumbling upon complaints on café.

I ran into these issues over four years ago, discussed them at the time, and the general consensus I got at the time was that these issues weren't going to be resolved, so I didn't follow up. I've seen nothing in recent release announcements to imply otherwise.

Really? [1] has quite a few things and I even made a visual guide to each change which was quite widely circulated. IIRC it was on main list, cafe, Planet Haskell, reddit, circulated on Twitter, posted in #haskell… My nginx says the post demonstrating the changes had ~3000 hits. I don't know what more I can do to make people aware of the releases.

To test this out, I put together a simple demonstration of a Hamlet synopsis in Markdown[1], and then converted it via Pandoc to Haddock[2]. I copied that Haddock content into a cabal file's description, then indented it, ran cabal haddock, and got:

cabal: hamlet.cabal:30: unexpected span: "!"

Cabal problem, no idea why it happens however, I have not looked into it.

For some reason, the trailing exclamation point is rejected by cabal. I tried removing the exclamation point, and it *did* generate content. However, all of the content was on the same line, because I needed to add periods between each block to deal with cabal's description parsing.

Also Cabal problem but you raise an important issue. Cabal's synopsis parsing sucks so if it could instead read from a separate file without these weird limitations, that'd be a big improvement in itself and a lot easier to automated if one wants to.

After adding those periods, things *mostly* worked, except that my first code sample came out as:

<p>Hi, my name is #

Notice how the {name} portion has been stripped away. So I see a number of ways that this current situation for writing synopses is inferior:

Maybe this is to do with Haddock issue #308 which was fixed in June and made it into 7.8.3 release. In fact, here[2] is the AST the parser (found in ‘haddock-library’ package which you can trivially install and try yourself) produces. This means I have to ask for your Haddock version. If you're on 7.8.2 or 7.8.3 you should be able to upgrade to 2.15.0 (that's right, we had another release since) simply by cabal install haddock which should demonstrate the correct behaviour. I'm surprised that you have a version without this fix unless you're stuck on 7.6.3 or something.

1. Far more people are comfortable writing Markdown than writing Haddock, even those of us who write a lot of Haddock. Tooling support for editing Markdown is also superior.

As usual, I point out that there are tens of different Markdown flavours. Of course today I will be pointed to the recent common Markdown creation but as far as I'm concerned, we had 14 standards and this is the 15th until it gets established. More importantly, Markdown is not suited for documenting code. We'd have to introduce new identifiers if we wanted to link to some functions from our synopsis. 16th standard aside, this is quickly starts looking like Haddock. I'm pretty sure that if you only used Haddock features that Markdown also has, you could tell someone your Haddock is a flavour of Markdown and they'd believe you. In any case, I'm not here to wage war against Markdown people but to find out why your Haddock experience is bad so let's move on. Bah, regarding tooling, with the parser now being in its own small package it's trivial to get out the AST for any snippet of text. If someone is eager then they can write emacs modes or whatever people use pretty easily now.

2. Having the synopsis in a separate file means that it's easy to give a link to the file on Github and get readable docs, which is *not* the case with linking to a cabal file.

I am not against this at all, I would quite like to see a separate file, it would make for much easier editing and you don't have to post-process anything spit out by tools to conform to additional cabal gotchas.

3. I don't have to go through any preprocess step to convert from Markdown to Haddock if that's the way I want to edit.

Seems like something cabal could handle for you if it sees pandoc on your system.

4. Putting the synopsis in the cabal file means we have to deal with cabal rules *in addition to* Haddock rules, such as the periods between blocks and not having an exclamation point at the end of the line[3].

Solved by your point 2.

5. On top of all that, Haddock still can't handle the documentation I want to write for Hamlet, to demonstrate #{variable} interpolation.

As the AST at [2] demonstrates, it can. Bah, just for you I used your pandoc-generated Haddock, stuck it in a module and produced the HTML file at [3]. It looks fine to me. We probably want to teach pandoc that it can put the anchors on the same line as the headers and that it can put content on the very next line from the header which would remove excessive spacing but other than that, it looks fine to me. Notably your #{name} problem is not present. [4] houses the Haskell file used to generate the HTML. Generated with Haddock HEAD but it only is 4 commits ahead of 2.15.0 release with the only changes being some documentation stuff.

So again, I really like the idea of doing the same thing for synopses as we have already for changelogs.

I also do, I am here merely to investigate why Markdown is so strongly preferred when Haddock does the job. Yes, the escaping was really bad in the past. Yes, it had some weird bugs. Yes, it has been fixed. To my knowledge, we have zero bugs open on the issue tracker[5]. There are some that are to do with GHC lexer + parser but those are not relevant at all here. If you know of bugs then please report them.

Michael

[1] http://lpaste.net/111103 [2] http://lpaste.net/111104 [3] For the record, I was completely unaware of this limitation, and stumbled upon it by accident when putting together this example.

...

I think just about the only thing nowadays that might surprise people is that tokens stretch over newlines but this is by design, especially considering one can embed code.

...

I would much rather be able tom write a synopsis in markdown, or asciidoc, or HTML, rather than haddock markup.

As I mention on the other e-mail, you can now use pandoc to achieve this. If people really care then it should not be difficult to co-ordinate with cabal + Hackage and make it parse your favourite format and splice it into synopsis.

I'd quite like the complaints about the existing Haddock markup, the rules are pretty damn close to what Markdown uses.

-- Mateusz K. _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

[1]: https://github.com/haskell/haddock/blob/master/CHANGES#L55 [2]: http://lpaste.net/111124 [3]: http://fuuzetsu.co.uk/misc/md/MarkdownToHaddock.html [4]: http://lpaste.net/111125 [5]: https://github.com/haskell/haddock/issues -- Mateusz K.