Hi All, During the last few days I've been working on the ModuleExtractor - a high level extractor of modules from the Haskell source files. This is not a low level parser -- as used by the compilers -- since it only cares for the things related to documentation. I am using the Daan's Leijen Parsec library, which seems to be well designed, documented and reasonably fast. The motivation for this work is to replace my home brewed parsing of source files in Haskell Module Browser (or rather a sophisticated "grepping") - which I currently do in Smalltalk - by a Haskell version. I do not believe that I will gain much on speed here (Hugs implementation will be probably much slower than the Squeak's one) but the idea is to move as much code as possible from the Squeak's to the Haskell's side in order to create a support code which could benefit other people wishing to interface such browsers to systems other than the Squeak. I think this information is relevant to our discussion and could help in clarifying some issues and provide some experimental tool. The parser aims to extract this information from the source files: data Module = Module { name :: String -- done , comment :: String -- done , exports :: [Export] -- chunk for now , imports :: [Import] -- chunk for now , fixities :: [Fixity] -- done , classes :: [Class] -- chunk for now , instances :: [Instance] -- chunk for now . categories :: [String] -- chunk for now , functions :: [Function] -- done , footnote :: String -- done } At the first stage, the parser breaks the source code into chunks: type Chunk = [Comment, Code] and then examines each chunk to convert it to one of the above specified entities. For example, the Function datatype is defined as: data Function = Function { funName :: String , funSignature :: Signature , funBody :: String } The good news is that the parser is able to deal with any positional placement of comments. For example, when it deals with functions it considers any one or all (concatenating all of them) the following comment options: + Many "--" comments or "{- .. -}" comment before the signature x Signature + Many "--" comments or "{- .. -}" comment after the signature x First line of function body + Many indented "--" comment lines x Indented function body Similar pattern applies to other entities. But in order of this positional approach to work I had to admit a concept of a category (known and cherished in Smalltalk, Objective C, Eiffel). In Haskell case, a special banner separates groups of functions. If this is not indicated somehow then the banner would become a part of the comment of the entity that follows it (wrong, but not catastrophic). It seems, after all, that I was not entirely correct in one of my previous posts - an intelligent parser can cope with a purely positional layout, given a bit of help related to definition of category delimiters. I should have remembered this, because I've done similar parsing for Xcoral browser for Java. I thought that this would be a helpful information for our discusion. I'll post the code when it's ready. Jan

Malcolm wrote:

Well I for one would suggest that it is better to announce the list first, before making decisions. The alternative is for us to have a long discussion, then immediately after announcing the list, to have the whole discussion again, when new people disagree with all the decisions that have already been taken. :-)

OK, maybe it's just as well to announce it then. What does Armin think? Here's a first shot at an announcement: ----------------------------------------------------------------------- At the recent Haskell Implementors' meeting in Cambridge, UK, it was decided that it would be useful to have a standard for embedded Haskell documentation. Such standards, and associated tools for extracting and formating the documentation in various ways, exists for other languages like Java and Eiffel and have proven to be very useful. Some such tools also exist (and are being actively developed) for Haskell, but there is as yet no generally agreed upon standard for the format of the embedded documentation as such. To address this, a mailing list has been started with the aim of defining a standard for embedded Haskell documentation, and possibly also related standards which would facilitate the development of various tools making use of such documentation (formatters, source code browsers, search tools, etc.). We feel that it is important to involve all who might be interested in this work at an early stage, so that as many aspects as possible can be taken into consideration, and so that the proposal for a standard which hopefully will emerge has a reasonable chance of gaining widespread support. Thus, You are hereby cordially invited to join haskelldoc@haskell.org. To join, just goto http://www.haskell.org/mailman/listinfo/haskelldoc. ----------------------------------------------------------------------- What do you think? Is something like this good enough and what we'd like to say? /Henrik -- Henrik Nilsson Yale University Department of Computer Science nilsson@cs.yale.edu

Henrik Nilsson wrote:

Here's a first shot at an announcement: ....

Yes, the proposed announcement looks good to me, and you can add my signature to it if you wish. Regards, Malcolm

I produced two samples of "long interface", extracted by module Extractor and formatted by my new module InterfacePrinter. Both samples interface to the latter. One is an unformatted ascii version (yyy.txt) and the other (yyy.html) is html formatted. The latter is intentionally link-free, but the cross references can be produced with ease if needed, since Extractor produces well structured data representing the module. Both samples can be found in http://www.numeric-quest.com/haskell/. I edited the html version a bit, adding few items for completness (exports, imports, classes, instances). The reason for this is that those edited items are temporarily treated as strings, and not structured as yet by Extractor. But the rest has been automatically generated and -- aside from the edited part -- both versions are factually identical. I wish to make the following points: 1. No heavy formatting information is stored in the source file from which the interface has been extracted. However, there are three hints for html beautifier from InterfacePrinter module. a. Those words in function comments which are intentionally embedded in single quotes, refer to function arguments. The printer uses this hint to produce italic emphases for html version. This hint is also used to prettyfy module comment (top of the page) as well, in order to emphasize certain keywords. b. Similarly, double quotes could be used as hints to produce bold emphases. However, I do not use it so far. c. The last hint signifies headers of groups of functions, which I call "categories". The hint is a comment that starts with "--: ". All other delimiters are stripped off. As an example, the banner: --------------------------------- --: interface listings - exported --------------------------------- will be replaced by a single line of this category header: "category: interface listing - exported". This line will be printed in bold when using html format. In summary, the formatting information embedded in the sources is very lightweight and does not compromise readability of sources. I would even argue that this helps to write well structured comments. The cycle: write and document a module, extract its interface, try to decide whether such interface is able to stand on its own and represents readable piece, go back and edit your comments again ... should certainly produce well structured and readable documentation. 2. Ascii version has its own merits too. For example, by typing in Hugs Prelude> f "someFunction" "someSource" shortList Ascii you should see the interface of function "someFunction" extracted from "someSource". This includes the function signature and comments. Similarly, you can browse the full listing of this function by specifying: codeList instead of shortList. Isn't that neat? But this is just a side effect. Function 'f' has been designed for random testing of module Extractor. Variety of interface types can be produced, once both modules are ready. I list them in the footnotes of yyy.txt or yyy.html. So far, the "long interface" (all entities, including private ones) is roughly defined. Regards, Jan