Hi

One idea that does strike me is that it would be super useful to have the ability in ghci to extract the haddocks associated with a function.

> :doc map

would result in:

-- | 'map' @f xs@ is the list obtained by applying @f@ to each element -- of @xs@, i.e., -- -- > map f [x1, x2, ..., xn] == [f x1, f x2, .. -- > map f [x1, x2, ...] == [f x1, f x2, ...]

That will be in Hoogle 4, as it does already directly relate to what Hoogle has to do. As for doctest, i think that could be implemented as a --warn flag to Haddock, and might be a useful thing to have. For example "90% of the base library has documentation, please go add documentation to 'sinh'", as a warning message. Thanks Neil

Thanks for the quick responses. To elaborate on how doctests work in python: (Background: in python functions, classes & modules are all "objects" -- having attributes as well as (for functions) the ability to run them. They all have, in particular a documentation string associated with them. To get at these doctest module just loads the module, then examines its contents, looking for doc strings of contained functions & classes... so the language itself provides introspection capabilities: a strong point of python. Maybe Haskell should also have this :).) Doctests in python are bits of code in documentation strings specially annotated. In the docstring for a function "f", we might have (with apologies for python function call syntax :)):

...

...

f( foo ) bar

This means: run f( foo ) and check that (the string representation of ) the output is "bar". You can put any code in docstring. In effect, the doctest module turns what you have written into a little module, together with imports, etc. For continuation lines on blocks of code the convention is:

...

...

f( ... foo )

A blank line after a ">>>" line means that no output is expected (you defined a function, for example). If a blank line is part of the string representation of output, then it is represented, e.g.:

...

...

f( foo ) <BLANKLINE> bar

If result has lots of text, you can use

...

...

f( foo ) <ELLIPSIS> bar

This is helpful, e.g. if f is a helper function that catches an exception then prints it out; you don't want to check the resulting stack-trace -- just the part at the end about what exception was caught. There are lots of other switches & options, but no need to copy all of the details. (Anyway, by the time I get to worrying about them, I'll have my own opinions. :)) --------------------------------------- So Paul Johnson writes:

You need to support QuickCheck and Hunit

Haddock does not contain a full Haskell parser, which makes it difficult to extend in a principled way (or maybe I'm just not a good enough

In fact, at least in python, doctests are even more generic... They support "anything" -- though QuickCheck tests are also allowed. Does this approach seem ok/desirable here too? A bit worrisome info from Paul: programmer).

I also looked at using the Haskell parser in the libraries, but that doesn't capture comments. Associating a comment with a particular construct is not that simple either.

Hmm... this makes me think that the first thing would be for me to put in an option to capture comments in the parse tree in the parser. Then Paul and crew would hack the parser's use into Haddock. Then I'd extend Haddock to support doctests! How about that? Or actually I think its overkill. If I can hook into the processing of the comment (presumably only extended comments), I can piece together a fake "module". The "..." convention means I should know how to stitch together expressions without having to understand them. One problem is checking the "output" of an expression. If an expression has output, it would seem I should throw in a test (using QuickCheck?). Hmmm. - Shaun

-----Original Message----- From: Thomas Schilling [mailto:nominolo@googlemail.com] Sent: Saturday, March 22, 2008 9:54 AM To: Neil Mitchell Cc: Don Stewart; Shaun Cutts; haskell-cafe@haskell.org Subject: Re: [Haskell-cafe] "doctest" for haskell -- a good project?

On 22 mar 2008, at 13.17, Neil Mitchell wrote:

...

Hi

...

One idea that does strike me is that it would be super useful to have the ability in ghci to extract the haddocks associated with a function.

...

:doc map

would result in:

-- | 'map' @f xs@ is the list obtained by applying @f@ to each element -- of @xs@, i.e., -- -- > map f [x1, x2, ..., xn] == [f x1, f x2, .. -- > map f [x1, x2, ...] == [f x1, f x2, ...]

That will be in Hoogle 4, as it does already directly relate to what Hoogle has to do.

As for doctest, i think that could be implemented as a --warn flag to Haddock, and might be a useful thing to have. For example "90% of the base library has documentation, please go add documentation to 'sinh'", as a warning message.

Though that would be a nice feature, too, I think you misunderstood what a doctest is.

A doctest is a unit test in the documentation. This way it serves as a usage example as well as a formal specification of the semantics. I think we already have some way to run quickchecks inside documentation, but I can't remember what it was that could do that.

/ Thomas

Hey Don, On Sat, Mar 22, 2008 at 8:39 AM, Don Stewart wrote:

I'm not sure how doctest works, or how it would work in a Haskell setting, could you elaborate?

In a nutshell, Python doctest has the programmer put an example "interactive session" in a functions docstring. A doctest module then extracts those, tries running the function on the inputs and sees if it matches the output. Best shown by an example: def adder(a,b): """Returns the sum of the two arguments. >>> adder(10,10) 20 >>> adder("a","b") 'ab' >>> adder(10,"100") Traceback (most recent call last): File "test.py", line 6, in adder return a+b TypeError: unsupported operand type(s) for +: 'int' and 'str' """ return a+b if __name__ == "__main__": import doctest doctest.testmod() # Test this module Running this script with $ python test.py -v gives the following output Trying: adder(10,10) Expecting: 20 ok Trying: adder("a","b") Expecting: 'ab' ok Trying: adder(10,"100") Expecting: Traceback (most recent call last): File "test.py", line 6, in adder return a+b TypeError: unsupported operand type(s) for +: 'int' and 'str' ok 1 items had no tests: __main__ 1 items passed all tests: 3 tests in __main__.adder 3 tests in 2 items. 3 passed and 0 failed. Test passed. I.e it extracted the test cases from the sample interactive session and compared the real results.

One idea that does strike me is that it would be super useful to have the ability in ghci to extract the haddocks associated with a function.

> :doc map

would result in:

-- | 'map' @f xs@ is the list obtained by applying @f@ to each element -- of @xs@, i.e., -- -- > map f [x1, x2, ..., xn] == [f x1, f x2, .. -- > map f [x1, x2, ...] == [f x1, f x2, ...]

marked up in ascii.

I'm not sure if that's related to doctest, but it sure would be useful!

Python does exactly that and it is very very useful. I'd love to see that in GHCi. Arnar

In a nutshell, Python doctest has the programmer put an example "interactive session" in a functions docstring. A doctest module then extracts those, tries running the function on the inputs and sees if it matches the output. Best shown by an example:

By the way, python does this by creating a temporary module from the docstring, then evaluating this. Is there a dynamic way in Haskell to evaulate a module? Or will I have to be content starting some sort of "subsession"... Which will make the doctest module less robust & portable. One idea would be just to produce a set of modules in the "core" doctest module. Then there would be a set of lightweight scripts for automating "produce & run" in various environments. But if there is a good "internal" way that I don't know about, I'm all ears. - Shaun

Hi

I once looked at doing this, but I didn't get very far.

Me too, and I managed to get some way:

resemble a formal specification. For a couple of examples, see my RangedSet package and Neil Mitchel's FilePath package. I manually copied the RangedSet tests into the Haddock documentation, while Neil wrote a small Haskell script to extract his tests from his documentation automatically. Unfortunately his script is tied to specific aspects of his FilePath package.

Yes, the problem was that FilePath wants to execute half the tests twice with different modules, and half with just one namespace. As far as tests go, this is a very wacky requirement. I wanted to generalise it into a tool, but couldn't find a sensible generalisation that still worked with filepath - and hence didn't bother. I think the solution is to not permit the quirkyness of filepath, and write a general solution for everything else. As someone who has frequently considered writing this, even going as far as brainstorming on a whiteboard, I would be an enthusiastic user of this. I think the lack of this tool in Haskell is a big hole which someone needs to fill. I particularly like the facility in FilePath: -- > x == reverse (reverse x) -- > reverse "neil" = "lien" i.e. I can write both quickcheck properties (quantified over all single letter variables), or I can write an instance (like doctest in Python) Thanks, and good luck! Neil

Hi

Niel -- I understand your script is part of FilePath... might it be a good starting point for abstraction? Can you point me to it?

You are certainly welcome to start from it, if it is of any use to you: darcs get http://darcs.haskell.org/packages/filepath/ Then look in the test directory.

Thank you for the support... This might take me a while, I must warn, as this is an "after work" project, and I am a consultant, so "after work" often doesn't come :). But its certainly motivating to work on something right away that could be useful (to myself as well).

I will be greatful for anything that comes out - and if you don't get anywhere with the code but do come up with a design, you can always write a blog post on it so if it ever gets continued people can start from where you left off. Thanks Neil

On Sun, Mar 23, 2008 at 5:09 PM, Marc Weber wrote:

Adding documentation ficilities to ghci is nice, however my experience is that documentation is not complete everywhere. That's why I'm looking at source code directly (thus having doc strings if given else I can have look at source directly).

This reminds me of a very handy feature in ipython (a custom python repl) - you can type name? which prints the docstring associated with name (which can be a function, module, class etc.) - but if you type name?? you get the docstring _and_ the source code (colorized and all :). I use it a lot when I'm experimenting in a interactive session. I could see something similar for haskell being very useful, to me at least. Arnar