How to browse code written by others

Hello all, I need your advice about how to browse code which was written by someone else (Paul Hudak's Euterpea, to be precise, apx. 10000 LOC). I had set some hopes on leksah, and it indeed shows me the interfaces, but I have not yet convinced it to show me more than that. I ran haddock over the sources, and again I could not see more that just signatures. I would be very happy with something like a Smalltalk browser. Something that would let me zoom down to the source code, but with "search" and hyperlink capabilities ("senders" and "implementers" in Smalltalk). Anyways, how do you guys do it, i.e. how to you dive into non-trivial foreign code? -- Martin

Hi Martin With Haddock you can also make the highlighted source as per docs hosted on Hackage. Install hscolour, then when you invoke haddock on the cabal file add the --hyperlink-source flag. No hyperlinks in the source though, just colourization. Thomas Hallgren had another source-to-html tool with hyperlinks in the html source rather than just colourizing, but I'm not sure what happened to it - maybe it is part of Programatica? http://ogi.altocumulus.org/~hallgren/Programatica/ Best wishes Stephen

Excerpts from Martin Drautzburg's message of Sun Jun 13 22:32:18 +0200 2010:
Anyways, how do you guys do it, i.e. how to you dive into non-trivial foreign code? I use Vim and tag files generated by Vim.
Using gnu idutils or such you can find any occurences of words very fast in large code bases. Of course they are not language aware but they often get the job done. I'd like to share all my scripts. Write me an email if you're interested. Marc Weber

Hello, On 13.06.2010, at 22:32, Martin Drautzburg wrote:
I need your advice about how to browse code which was written by someone else (Paul Hudak's Euterpea, to be precise, apx. 10000 LOC). I had set some hopes on leksah, and it indeed shows me the interfaces, but I have not yet convinced it to show me more than that.
I ran haddock over the sources, and again I could not see more that just signatures.
I would be very happy with something like a Smalltalk browser. Something that would let me zoom down to the source code, but with "search" and hyperlink capabilities ("senders" and "implementers" in Smalltalk).
Anyways, how do you guys do it, i.e. how to you dive into non-trivial foreign code?
I use the following tools: * haddock generated docs with hyperlinked sources * MacVim (or just vim) with Claus Reinke's haskellmode-vim, see: http://projects.haskell.org/haskellmode-vim/index.html Have a look at the screencasts to see documentation lookup, and code navigation: http://projects.haskell.org/haskellmode-vim/screencasts.html Make sure you know how to use tags inside of vim. ghci is able to generate the tagsfiles for you. This allows you to jump to definitions of identifiers. * SourceGraph, it generates an HTML report of a cabal projekt or of any source tree. IMHO, great to get the overall picture. Regards, Jean-Marie

On Mon, Jun 14, 2010 at 2:02 AM, Jean-Marie Gaillourdet
Hello,
On 13.06.2010, at 22:32, Martin Drautzburg wrote:
I need your advice about how to browse code which was written by someone else (Paul Hudak's Euterpea, to be precise, apx. 10000 LOC). I had set some hopes on leksah, and it indeed shows me the interfaces, but I have not yet convinced it to show me more than that.
I ran haddock over the sources, and again I could not see more that just signatures.
I would be very happy with something like a Smalltalk browser. Something that would let me zoom down to the source code, but with "search" and hyperlink capabilities ("senders" and "implementers" in Smalltalk).
Anyways, how do you guys do it, i.e. how to you dive into non-trivial foreign code?
I use the following tools:
* haddock generated docs with hyperlinked sources * MacVim (or just vim) with Claus Reinke's haskellmode-vim, see: http://projects.haskell.org/haskellmode-vim/index.html Have a look at the screencasts to see documentation lookup, and code navigation: http://projects.haskell.org/haskellmode-vim/screencasts.html Make sure you know how to use tags inside of vim. ghci is able to generate the tagsfiles for you. This allows you to jump to definitions of identifiers.
If you go this route, I will shamelessly promote hothasktags instead of ghci. It generates proper tags for qualified imports.
* SourceGraph, it generates an HTML report of a cabal projekt or of any source tree. IMHO, great to get the overall picture.
Regards, Jean-Marie
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

On 15.06.2010, at 01:35, Luke Palmer wrote:
On Mon, Jun 14, 2010 at 2:02 AM, Jean-Marie Gaillourdet
wrote: Hello,
On 13.06.2010, at 22:32, Martin Drautzburg wrote:
I need your advice about how to browse code which was written by someone else (Paul Hudak's Euterpea, to be precise, apx. 10000 LOC). I had set some hopes on leksah, and it indeed shows me the interfaces, but I have not yet convinced it to show me more than that.
I ran haddock over the sources, and again I could not see more that just signatures.
I would be very happy with something like a Smalltalk browser. Something that would let me zoom down to the source code, but with "search" and hyperlink capabilities ("senders" and "implementers" in Smalltalk).
Anyways, how do you guys do it, i.e. how to you dive into non-trivial foreign code?
I use the following tools:
* haddock generated docs with hyperlinked sources * MacVim (or just vim) with Claus Reinke's haskellmode-vim, see: http://projects.haskell.org/haskellmode-vim/index.html Have a look at the screencasts to see documentation lookup, and code navigation: http://projects.haskell.org/haskellmode-vim/screencasts.html Make sure you know how to use tags inside of vim. ghci is able to generate the tagsfiles for you. This allows you to jump to definitions of identifiers.
If you go this route, I will shamelessly promote hothasktags instead of ghci. It generates proper tags for qualified imports.
That sounds interesting. Thanks for the hint. Regards, Jean-Marie

..ghci is able to generate the tagsfiles for you. This allows you to jump to definitions of identifiers.
If you go this route, I will shamelessly promote hothasktags instead of ghci. It generates proper tags for qualified imports.
What do you mean by "proper" here? GHCi has the information needed to generate more detailed tags, but the tags file format did not support much more detail last time I checked. Tags for explicitly qualified names could be generated (and probably should be), though that would interact with the default identification of Haskell identifiers in Vim. But if you want to resolve imports properly (or at least a bit better, such as adding "import qualified as" support, or pointing unqualified uses to the correct import), you need more support from the tags mechanism. There is a tags file format proposal here: http://ctags.sourceforge.net/FORMAT that does (among other scopes) suggest explicitly file-scoped local tags file: Static (local) tag, with a scope of the specified file. When the value is empty, {tagfile} is used. but in Vim 7.2 the help file still says http://vimdoc.sourceforge.net/htmldoc/tagsrch.html#tags-file-format The only other field currently recognized by Vim is "file:" (with an empty value). It is used for a static tag. If Vim somehow started supporting that extended file:<scope> format without updating its docs, that would be good to know (what version of Vim? where is this documented?). Your suggested use-case for such a feature is interesting, but we're getting into an area where "live" calls to a scope resolution tool might make more sense. If one is willing to depend on Vim's Python-binding, one could keep a GHCi session live in the background, track the current file/module, and use the ":info" "-- Defined at" output to find the correct definition. Btw, GHCi's ":browse!" gives information on where available names come from, which can be useful for resolving unqualified names (which "map" is that?) in unknown code. Claus

Hi Claus, On 06/15/2010 05:57 PM, Claus Reinke wrote:
If you go this route, I will shamelessly promote hothasktags instead of ghci. It generates proper tags for qualified imports.
What do you mean by "proper" here?
I think Luke means that if you use qualified names then hothasktags can give you better location information than current ghci ctags. I discussed this with Luke before and I sumarrized what would need to be done to imporove ghci ctags to support qualified names better. Here is the post which explains it with an example: http://permalink.gmane.org/gmane.comp.lang.haskell.cafe/73116 With this addition, I believe, ghci tags would at least as good as hothasktags. That said I do not try hothasktags. Do hothasktags do analyzes deep enough that the information provided is always correct? I did not go to add this to ghci ctags since I barely ever use qualified names so it is a non-issue for me. Also propper support for scoped tags would include some vim macro which (on ^] key-press) tries to find a qualified name first and only after a failure it would try to find plain name without qualification. So if one wants to use it well he/she needs first select the name in visual mode and only then pres ^]. (Or one should use full ex commands for navigation like :tselect.)
but in Vim 7.2 the help file still says
http://vimdoc.sourceforge.net/htmldoc/tagsrch.html#tags-file-format
The only other field currently recognized by Vim is "file:" (with an empty value). It is used for a static tag.
If Vim somehow started supporting that extended file:<scope> format without updating its docs, that would be good to know (what version of Vim? where is this documented?).
I did not find it documented but it looks like a filename can be used in place of <scope>. If provided then the validity of the tag is limited to the filename.
Your suggested use-case for such a feature is interesting, but we're getting into an area where "live" calls to a scope resolution tool might make more sense.
Och I would like to have a "vim" with full incremental parser for Haskell ... with things like AST being drawn in a pane, intellisense and completion on symbols but also on grammar/syntax, re-factoring options and all the goodies this could provide. It will not be a vim any more. Probably won't happen in my lifetime either but I can dream :) Peter. PS: Thanks for your vim support.

If you go this route, I will shamelessly promote hothasktags instead of ghci. It generates proper tags for qualified imports. What do you mean by "proper" here?
I think Luke means that if you use qualified names then hothasktags can give you better location information than current ghci ctags.
GHCi :ctags doesn't output tags for qualified names (though it probably should), but that isn't enough for proper handling of qualified imports. I believe hothasktags is meant to output multiple tags per target, one for each module referencing the target. But that would rely on scoped tags.
I discussed this with Luke before and I sumarrized what would need to be done to imporove ghci ctags to support qualified names better. Here is the post which explains it with an example:
http://permalink.gmane.org/gmane.comp.lang.haskell.cafe/73116
The problem with that example is that all occurrences of B.x point to b.hs and C.x always points to c.hs, so it doesn't test the scoping aspect of static tags. For instance, if you are in b.hs, and try ':tag C.x', you'll still be sent to c.hs, even though that isn't in scope (see also ':help tag-priority'). If I add a file d.hs that is the same as a.hs but with the qualifiers exchanged: module D () where import qualified B as C import qualified C as B localAct = do print B.x print C.x and try to add corresponding scoped tags by hand, then I don't see the scoping being taken into account (though my Vim is old 7.2 from August 2008). Depending on sort order of the tags file, either all B.x point to b.hs or all B.x point to c.hs. So, one either gets the wrong pointers in a.hs or in d.hs.
I did not go to add this to ghci ctags since I barely ever use qualified names so it is a non-issue for me. Also propper support for scoped tags would include some vim macro which (on ^] key-press) tries to find a qualified name first and only after a failure it would try to find plain name without qualification. So if one wants to use it well he/she needs first select the name in visual mode and only then pres ^]. (Or one should use full ex commands for navigation like :tselect.)
One could redefine 'iskeyword' to include '.', but that may not always be what one wants. haskellmode also uses a script, though not yet for ^].
Your suggested use-case for such a feature is interesting, but we're getting into an area where "live" calls to a scope resolution tool might make more sense.
Och I would like to have a "vim" with full incremental parser for Haskell ... with things like AST being drawn in a pane, intellisense and completion on symbols but also on grammar/syntax, re-factoring options and all the goodies this could provide. It will not be a vim any more. Probably won't happen in my lifetime either but I can dream :)
The main thing standing in the way is Bram's strict interpretation of ':help design-not'. It seems to be an overreaction to Emacs as an OS, wanting to keep Vim small and simple. Ironically, adding one standard portable way of communicating with external tools would allow to make Vim smaller (removing all those special case version of tool communication). And semantics-based editing is wanted in all languages. So the feature has been rising and is currently topping the votes:-) http://www.vim.org/sponsor/vote_results.php Just keep Vim for what it is good at, and add a way to interface with language-aware external tools doing the heavy lifting (the old Haskell refactorer had to use sockets for the Vim interface, the Emacs interface was rather simpler in that respect). Claus
PS: Thanks for your vim support. You're welcome.

On 06/20/2010 11:05 AM, Claus Reinke wrote:
I think Luke means that if you use qualified names then hothasktags can give you better location information than current ghci ctags.
GHCi :ctags doesn't output tags for qualified names (though it probably should), but that isn't enough for proper handling of qualified imports. I believe hothasktags is meant to output multiple tags per target, one for each module referencing the target.
I understand it that way too.
But that would rely on scoped tags.
By scoped tags, I understand tags valid only in the given scope (which is a file for the sake of qualified imports) which is the same as static tags which are supposed to be valid inside a file too ... but they are not as I found out from your example later :-(
I discussed this with Luke before and I sumarrized what would need to be done to imporove ghci ctags to support qualified names better. Here is the post which explains it with an example:
http://permalink.gmane.org/gmane.comp.lang.haskell.cafe/73116
The problem with that example is that all occurrences of B.x point to b.hs and C.x always points to c.hs, so it doesn't test the scoping aspect of static tags. For instance, if you are in b.hs, and try ':tag C.x', you'll still be sent to c.hs, even though that isn't in scope (see also ':help tag-priority').
Yes, but this is the way vim works. Vim jumps even when given tag is not in scope. C.x is not in scope in b.hs (not as a static nor as a global symbol) so vim gives as static tag in a different file which matches exactly. Which looks like a reasonable feature for non-hierarchical symbols (maybe one is missing an include/import). Well, maybe not so much for jumps to tags but definitely usefull in :tselect. If we would use hierarchical names for tags then this is not so good and it works poorly (if I can say it works at all) for tag completions (^X^]).
If I add a file d.hs that is the same as a.hs but with the qualifiers exchanged:
module D () where import qualified B as C import qualified C as B localAct = do print B.x print C.x
and try to add corresponding scoped tags by hand, then I don't see the scoping being taken into account (though my Vim is old 7.2 from August 2008). Depending on sort order of the tags file, either all B.x point to b.hs or all B.x point to c.hs. So, one either gets the wrong pointers in a.hs or in d.hs.
You are correct. I thought this works well. Thanks for pointing it out. It does not work for me in my vim too, version 7.2 too. This looks to me like a vim bug. Though vim developers may not think so since it is not documented anyway. So it is really questionable how much do the qualified tags make sense in vim. I'm much less of a supporter of scoped tags now :-/ Looks like it is not worth the effort. Peter.

I've been using the geany http://www.geany.org/ editor recently and I was
shocked to find that it has decent source browsing capabilities (that work
with haskell even!). You can find where something is defined and find other
usages of things. It's a bit crude, but gets the job done well enough.
- Job
On Sun, Jun 13, 2010 at 4:32 PM, Martin Drautzburg wrote: Hello all, I need your advice about how to browse code which was written by someone
else
(Paul Hudak's Euterpea, to be precise, apx. 10000 LOC). I had set some
hopes
on leksah, and it indeed shows me the interfaces, but I have not yet
convinced it to show me more than that. I ran haddock over the sources, and again I could not see more that just
signatures. I would be very happy with something like a Smalltalk browser. Something
that
would let me zoom down to the source code, but with "search" and hyperlink
capabilities ("senders" and "implementers" in Smalltalk). Anyways, how do you guys do it, i.e. how to you dive into non-trivial
foreign
code? --
Martin
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
participants (9)
-
Claus Reinke
-
Edward Z. Yang
-
Jean-Marie Gaillourdet
-
Job Vranish
-
Luke Palmer
-
Marc Weber
-
Martin Drautzburg
-
Peter Hercek
-
Stephen Tetley