February 2001 - HaskellDoc

RE: Positional cues or not
by Simon Marlow 14 Feb '01

14 Feb '01

> One proposal was to allow both something like > > {--- > -} > > and > > -- -- > -- > ... > -- > > This is OK, but maybe a little too subtle. E.g. one might be forgiven > for wondering why not writing ---- or {- --. > > On the other hand, generalizing Simon's tagged proposal seems to > work well: > > {- @DOC > ... > -} > > and > > -- @DOC > -- > .. > -- Please - if we're going to adopt this kind of style, don't make it needlessly divergent from existing conventions. There's no reason to use {- @DOC -} instead of {-# DOC #-}. The end of line version could be: --# DOC -- Documentation goes here... -- But, some people clearly prefer a style which is easier on the eye, in which case the {--- -} and --- are fine by me. Cheers, Simon

7 14

RE: Positional cues or not
by Simon Marlow 13 Feb '01

13 Feb '01

> > The end of line version could be: > > > > --# DOC > > -- Documentation goes here... > > -- > > Sorry, that breaks Haskell'98. --# is a valid operator symbol. yes, I always loved that particular change in Haskell 98. not. > > But, some people clearly prefer a style which is easier on > the eye, in > > which case the {--- -} and --- are fine by me. > > Triple dashes are nice because they > * have low visual density (not too obtrusive) > * look the same for nested/end-of-line varieties without > breaking H'98 I think we're arriving at a consensus here. Anyone object? Cheers, Simon

2 1

RE: Positional cues or not
by Simon Marlow 13 Feb '01

13 Feb '01

> > Using simple {- -} is too hard to parse; the lexer can't > pass these on > > to the parser as tokens, because they can occur anywhere. > > I am not sure what you mean to imply here. This is only > something the Doc tool has to do, isn't it. The mere fact > that a comment immediately preceeds an (exported) type > signature could be enough of a cue to turn it into a > documentation comment. The point I was make (badly) is that in order to incorporate documentation support into a Haskell parser, the natural thing to do would be to teach the lexer about documentation comments, and pass them through to the parser as tokens. Positional cues make this harder, because the lexer has to know a lot more about the context in order to do its job. Annotating documentation comments using a different style, like {--- -} or ---, make the job easier again. To be clear, I'm not arguing *against* positional cues, I'm arguing *for* using a distinctive comment style for documentation comments. Of course, you don't have to do this using a Haskell parser. My feeling is that this is a good idea however, since the tool needs to understand some Haskell in order to properly hyperlink and pretty print the type signatures & source. > More problematic are general comments not associated with a > single definition that the programmer might want to have in > the generated documentation. > > I also tend to think that {-# DOC #-} is too heavy. fine. :) Cheers, Simon

8 11

RE: Documentatio markup
by Simon Marlow 11 Feb '01

11 Feb '01

> > > Jan propses to use conventions like 'xxx' for variable names > > > and "zzz" for emphasis (I think). > > > other lightweight emphasis conventions like _yyy_ or *zzz* > > > > one that springs to mind for me is @Name@, but I don't > really mind as > > long as it's non-ambiguous and doesn't cause parsing headaches. > > Would there be any difficulty in permitting _all_ of the common ASCII > emphasis styles? For instance, "name" and _name_ could be italic, > *name* could be bold, 'name' and @name@ could be hyperlinked bold, > or some such. In fact, it wouldn't really matter which style was > rendered in what manner - different tools could do it differently if > they chose. (Ok, except maybe hyperlinking would have to be defined.) The real problem is that you need at least three forms of hyperlinked markup: eg, what should @ForeignPtr@ link to - the type constructor, the constructor, or the module? I suppose there's always @...@, |...|, <...>, <<...>>, etc. Would this be too heavyweight: @module M@, @con C@, @class C@, @tycon T@? Perhaps the keywords could be left out when the meaning is non-ambiguous, as is usually the case with variable and function names. Cheers, Simon

3 2

RE: Positional cues or not
by Simon Marlow 09 Feb '01

09 Feb '01

> > I don't particularly care whether the convention is {--- -} > or {-# DOC #-} > > (with a slight preference for the latter, because it is consistent > > with existing conventions). > > I have a distinct preference against {-# DOC #-}. It is too visually > dense and distracting. Besides, the simpler {- -} is a perfectly > good convention already as well. > > I'm not sure whether to require the extra markup of {--- -}. For one > thing, as Manuel pointed out, people often use -- to end of line > comments in addition to nested comments, and I think we should be > able to be flexible there. However, if you think special markup to > distinguish documentation comments from normal comments is really > necessary, maybe a three-dash --- to end of line could serve as the > analog to {--- -}. Using simple {- -} is too hard to parse; the lexer can't pass these on to the parser as tokens, because they can occur anywhere. Using {--- -} is fine by me (with the restriction that they can only occur in the same places you can put a type signature). Cheers, Simon

2 1

Positional cues or not
by Simon Marlow 09 Feb '01

09 Feb '01

[ meta-note: can we start splitting up the discussion into separate topics, and use different subject lines for each? I'm finding it hard to keep track of a dicussion that is presented breadth-first. ] Henrik says: > 1. I agree with Malcolm and Jan that the documentation conventions > need to be lightweight. (I too dislike literate programming, > except possibly when the aim is to write a paper or a book.) > > However, I think that relying solely on positional cues might be > too constraining and (in te long run) inflexible. So personally, > I think HDOc/JavaDoc-like tags is a good compromise. I've been mildly concerned about the lean towards to positional cues myself, mainly because of the constraining nature. I'm a strong believer of "mechanism not policy", but I'm aware that all too often this is a cop-out from tackling the hard problem of policy. However in this case I think we should try to separate the two at least. My vote goes to a combination of tags and mild positional cues: eg. a specially-marked comment before a type signature might be understood as documentation. I don't particularly care whether the convention is {--- -} or {-# DOC #-} (with a slight preference for the latter, because it is consistent with existing conventions). Cheers, Simon

3 2

Documentatio markup
by Simon Marlow 09 Feb '01

09 Feb '01

Henrik writes: > To that, I also see a need to add some lightweight conventions > for markup of explanatory text. E.g. I'd like to be able to mark > variable names, emphasize a piece of text, and maybe include > small code fragments. Absolutely. We need a way to include a (hyperlinked) reference to a code entity from the documentation, and that means separate markup for type constructors, data constructors, and module names (and possibly in the future class names; at the moment they share the type constructor namespace), and if we're being really anal about it, variables/type variables. > Jan propses to use conventions like 'xxx' for variable names > and "zzz" for emphasis (I think). That's probably reasonable, > and I indeed use the 'xxx' convention in my own comments > sometimes. But one should be aware that this useage can conflict > with the normal meaning of the quote characters. In particular, > other lightweight emphasis conventions like _yyy_ or *zzz* > spring to mind. one that springs to mind for me is @Name@, but I don't really mind as long as it's non-ambiguous and doesn't cause parsing headaches. Cheers, Simon

2 1

restarting the discussion
by malcolm-hs＠cs.york.ac.uk 08 Feb '01

08 Feb '01

Hello to everyone who has joined the HaskellDoc mailing list. We had a little bit of discussion before announcing the list more widely, but everything now seems to have stopped dead. So it's time to get thoughts rolling again. Do check the list archive on haskell.org to see what has already happened. I'll start by declaring my interest in automatic documentation. I have a proposal (not yet published, sorry, but coming soon) to help organise the Haskell libraries, standard and non-standard. But libraries are no good if you don't know what they contain. Not everyone is either willing or capable to write high-quality documentation for their libraries. Hence, a tool which automatically derives some kind of documentation directly from the source code would be extremely useful as a starting point for exploring a new library. When I first saw Armin Groesslinger's HDoc, I got very excited because it generates very useful summary information about a library, without the programmer having to add a single annotation whatsoever. Or at least that's how it appears at first glance. My immediate aim is to have a tool that produces browsable documentation to be read by the user of a module, not necessarily an author or developer (i.e. external docs, not internal docs). Ideally, I want to see * what useful datatypes are exported; * the type signature of every exported function; * some textual description of each datatype; * some textual description of each exported function; Now obviously if (as all good programmers do :-) there are type signatures in the source code, it is relatively easy to extract them with a tool. Textual descriptions can also be extracted from the programmer's comments. But the programmer will write many kinds of comment - not all are intended for users - many describe the implementation, which is of no interest to the mere user. So we need to distinguish different styles of comments. This can be done lexically or positionally, and we need to agree a standard. I have another big requirement. The source code must remain readable as source code. I absolutely loathe so-called "literate programming" style, because it breaks this rule horribly. From my point of view, any new documentation standard must be as non-intrusive as possible. This almost immediately rules out XML-style tags I'm afraid. Other (less heavy) lexical conventions might be OK though, provided there are only a small number of them to be learnt. I am intrigued by the use of positional cues (e.g. a comment just before or just after a type signature) as a clever way of associating documentation with code, whilst avoiding extra syntax. For those who have not yet looked at Armin's HDoc, can I encourage you to do so, as a concrete example of how some of these ideas have been put into practice. He develops a "special" kind of comment, introduced by {--- rather than {-, and has some small lexical conventions that help to generate nice hyperlinked HTML. Also, look at Jan Skibinski's tool which captures comments by their position with respect to a signature. Both are linked from the haskell.org page on libraries and tools. Regards, Malcolm

4 4

Re: restarting the discussion
by Jan Skibinski 08 Feb '01

08 Feb '01

Oops, I mistakenly responded to a private copy of Henrik's message and erased the public one. Resending to the list. Sorry Henrik, but you started all of this. :-) ---------- Forwarded message ---------- Date: Thu, 8 Feb 2001 18:10:26 -0500 (EST) From: Jan Skibinski <jans(a)numeric-quest.com> To: Henrik Nilsson <nilsson(a)cs.yale.edu> Subject: Re: restarting the discussion On Thu, 8 Feb 2001, Henrik Nilsson wrote: > Jan, one question: what do you do/intend to do with *re*exported > entities? > > As explained in my previous mail, I think external documentation > should include everything a module exports, including the related > documentation, regardless of where the various entities were > originally defined. I agree. This is what is called a "flat interface" in Eiffel, which can be selectively made a bit bigger or a bit smaller. They have a "mother of all classes" called ANY (if I remember it correctly) which just brings too much noise in sometimes. So they turn if off. Something similar to our Prelude. But I am not ready for this yet. > I have to say that this already looks like a quite useful tool, > and it does show that one can go a long way with very lightweight > conventions. A few points, tough. > > I'd again like to take the opportunity to push for the idea of also > defining and intermediate format. The documentation samples Jan > have presented are all very good examples of the kind of documentation > renderings one might want. But on the other hand, I don't think > they cover the entire spectrum of reasonable presentations. > (I believe I have some support from at least Jan on this one.) Definitely. > This brings me my main concern: is the *style* of conventions > Jan proposes (not necessarily exactly those conventions he currently > use) flexible enough to: > > 1. support other reasonable renderings, > 2. avoid creating a feeling among users that the conventions are > "dictatorial" ;-) > > Regarding 2, I'm not entirely serious: by necessity, the conventions > must be fairly strict. But after browsing through the Eiffel > documentation Jan sent a pointer too, I personally felt that the > required conventions were a bit too much. Now, I certainly do not > wish to start a discussion about the Eiffel conventions. I'm just > mildly concerned that too strict requirements might get in the way > of getting wide acceptance for a standard. But maybe this is a > non-issue. The style I've shown is just a recommendation, not necesserily a requirement. You can write what you wish - a parser would not care less. I just found this kind of style tremendously readable. So take the "dictatorial" back, will you? :-) > Regarding 1, below are a few examples of things that I think are > reasonable. > > Please keep in mind that I too favour a fairly lightweight > convention, but I am willing to allow a little extra noise for the > sake of flexibility. To be concrete, I like the convention 'xxx' > to mark a piece of code (such as a variable name) in running text > which Jan proposes, I think one or two more such conventions > would be useful (e.g. _xxx_ for emphasis). On the other hand, > I don't necessarily see anything wrong with HDoc/JavaDoc-style tags > for marking large things: the extra piece of information provided > could be quite valuable. This markup is invented for better readability but it would not be absolutely required. A properly structured comment should stand on its own legs - without any help from formatting. As an example - any of the following excerpts will do the same job: 1-- ...from 'module' 2-- ...from module 'm' 3-- ...from module m Versions 1 and 2 are just easier to read when rendered. Now to your challenges: > ----------------------------------------------------------------------- > Example 1. > > I sometimes write functions which take tuples among their arguments > and/or return tuples, and I sometimes find it useful to document > each field separately. > > Returning a tuple immediately implies that it might be difficult > to start each documentation block with a sentence that describes > what the function returns (which seems to be your convention)? We have a beautiful keyword "where", which is perfect for such occassions. But I do not claim that writing good comments is an easy task. It is often a challenge. But think about it a bit.. Java style is just an easy way out: one writes the comments automaticaly, without even thinking about the future users. In contrary, when one has to structure the comment in most readable way one often finds a flow in ones solution. This is what happens to me sometimes... I am not up to your challenge in your example 1, since I do not fully understand its functionality. However, it could start the comment with something like this: -- Five-tuple (#1, #2, #3, #4, #5) of "whatever is -- your indented meaning of this tuple" ... blah -- where -- #1 - list of new top-level definitions [.. this and similar version example #1 cut out ..] > ------------------------------------------------------------------------ > Example 2: > > The lambda lifter in the previous example illustrates another problem: > how to deal with moands? NS happens to be a monad, so it is not > really correct to claim that the function "returns a five tuple". > > In Armin's HDoc, there is a separte tag for marking monadic > return values. (But I think they still renders as "Returns ..."?) > > In your case you seem to adopt the convention that functions > with return type (IO a) should be described in imperative terms. > This makes sense for IO and many other monads, but not necessarily for > all monads. > > Do we need more structure here, or are monads just such a general > concept that there is not much one can do about it? Good point. Who said that the allmighty "documentation commitee" is a bunch of lazy guys doing nothing? :-) > ------------------------------------------------------------------------ > Example 3: > > This is really a variation on 2. Consider combinator libraries. > Sometimes it makes sense to adopt special commenting conventions. I agree here. > Again, the Fudget GUI combinator library can serve as a useful example. > > The central abstraction in the Fudget GUI is the Fudget, which can > be understood as a component with one user-visible input and one > user-visible output. Such a component may or may not have a graphical > rendering in the form of some GUI widget. Each fudget also has an > invisible, low-level input/output pair which connects it to the outside > world. An example could be a text input field. On the input it > would accept strings which are shown as default values to the user. > On the output, a user-entered string would appear as soon as the > user presses return. > > The documentation for a function like > > foo :: a -> b -> F c d > > where (F c d) is a fudget with input of type d(!) and output of type c, > would be something along the following lines: > > DESCRIPTION > A foo fudget is a button that ... > INPUT: True or False to switch the button on or off under > program control. > OUTPUT: True when the button is on, False when the button is off. > ARGUMENTS: > xxx :: a, ... > yyy :: b, ... > > The point here is that describing foo as a function returning a Fudget > isn't very helpful. Instead special conventions approporiate for > the domainin question were adopted. Even if I agree that special conventions must be adopted sometimes, I could still attempt to raise to your challenge here :-) foo :: a -> b -> F c d foo xxx yyy -- Button fudget (obtained as a result of unknown -- to me foo operation on arguments 'xxx' and 'yyy'), -- whose i/o are: -- input - a program controlled boolean -- output - True if button is on (false otherwise, which is obvious and redundant) -- Assumption here is that user already knows all about the operational model Fudgets use. The low level documentation should not even attempt to solve ontological problems. If you write a vehicle user manual you may - at one point - safely assume that user knows what a clutch is. It's good if a single function is as self-contained as possible. Think for example about reading it in Haskell Module Browser. But sometimes nothing one can do without referring to a bit broader concept, such as a module or a library of modules. > > I'm currently involved in the development of a framework for > domain-specific languages called FRP (Functional Reactive Programming). > One of its incarnations is as a combinator library. A central > abstraction is (Behavior a b) which represents a "transformer" of > signals of type a to signals of type b. I.e. there are some similarities > to a fudget (F b a). Again, when we document a behaviour, it would be > nice tobe able to talk about its inputs and outputs (which furthermore > often happen to carry elements of tuple types). > > So, can we find conventions which are flexible enough to support > documentation of this type of code? I still believe we can. I went through the Hawk exercises once to document them in this style and I was quite happy with the results. But I do not see anything wrong with customary styles for special cases - providing that user is given a chance to grab the top ideas first. > > BTW, the Fudget library manual is available on line. It might be worth > checking it out. For example: > > http://www.cs.chalmers.se/Cs/Research/Functional/Fudgets/Manual/current/sma… > > A concrete example illustrating the points above is: > > http://www.cs.chalmers.se/Cs/Research/Functional/Fudgets/Manual/current/tog… I am familiar with the concepts but I never looked at Fudgets from documentation point of view. > ----------------------------------------------------------------------- > Example 4 > > Well, not an example actually, more of a "laundry list". I touched > on most of thses in my previous mail: > > * A possibility to include pictures does not seem entirely unreasonable. As an extra, why not? > * A convenient way of including a piece of code, e.g. a useage example. > <code> </code> tags? > * Explicit cross references Cf. the man-page "see also" style. Quite > useful to be able to refer the reader to closely related functions > in a large library, for instance. > * Hints for generating indices at various level of detail > (beginner's index, programmer's index, ...) Many of the cross references, indices, etc. can be captured from the module structure itself. If I capture all structures of interacting modules, as Extractor intends to do, then I could - in principle - generate a lot of references, couldn't I? But yes, there are always special cases to consider.. Regards, Jan

1 0

RE: A Haskell Documentation Standard
by Simon Marlow 05 Feb '01

05 Feb '01

> That means that classes and instances (and their functions) have to be > distinguished by giving the complete type of the class / instance > declaration, right? > > E.g. > > class X a b where > f :: a -> b > > instance X Person Frog where > f x = ... > > How do we avoid that the tool confuses the two version of "f" ? > An obvious way would be > > {-# DOC f INSTANCE X Person Frog ... #-} > > {-# DOC f CLASS X ... #-} For classes, the 'class' keyword isn't required: classes and type constructor share the same namespace, so an identifier beginning with an upper case letter is unambiguous. I hadn't considered the documentation of individual instances, but perhaps you need that for internal documentation (most of the time, the only documentation you need for an instance is that it exists). If documentation for instances is required, then yes, you have to give the full instance header in order to identify the exact instance. > (HDoc required similar "help" in early versions, but I changed that in > favour of considering positional information to reduce the redundancy > required in the annotations.) > > > I guess additional annotations are an unavoidable drawback when not > relying on positional information. On the other hand, being > able to put > the documentation in different files may be a big advantage. > > So, should we allow both variants? I.e. use positional > information when > the pragma happens to be next to a class/instance declaration (or a > function therein) and rely on extra information (like "CLASS > X") in the > other case? I wouldn't object to allowing both variants - perhaps the convention could be that if the identifer is missing, then it applies to the following declaration. PS. Henrik - before this discussion gets too detailed, do you want to announce the existence of the mailing list on haskell(a)haskell.org? Cheers, Simon

6 9