My Haskell experiments have reached a size in which debugging tools would be more than welcome, so I looked around, and was very disappointed. I tried Hood, which is a pain to use (lots of editing of the code required), I looked at Buddha but didn't want to downgrade to GHC 5 for trying it (nor is my code Haskell 98, because of multi-parameter classes), and all that seems left to try is Hat, whose Web site I can't reach at the moment. So what are you Haskell programmers using for debugging in real life? Konrad.
On Tue, Aug 26, 2003 at 05:27:11PM +0200, Konrad Hinsen wrote:
My Haskell experiments have reached a size in which debugging tools would be more than welcome, so I looked around, and was very disappointed. I tried Hood, which is a pain to use (lots of editing of the code required), I looked at Buddha but didn't want to downgrade to GHC 5 for trying it (nor is my code Haskell 98, because of multi-parameter classes), and all that seems left to try is Hat, whose Web site I can't reach at the moment.
So what are you Haskell programmers using for debugging in real life?
I mostly just use Debug.Trace, which is basically like the printf method of debugging in C/C++ (which is what I usually use in those languages). It's not very elegant, but generally gets the job done. The other debugging tool (in a sense) I use is ghc's profiling, which I find helpful for figuring out scaling problems. -- David Roundy http://civet.berkeley.edu/droundy/
My Haskell experiments have reached a size in which debugging tools would be more than welcome, so I looked around, and was very disappointed. I tried Hood, which is a pain to use (lots of editing of the code required), I looked at Buddha but didn't want to downgrade to GHC 5 for trying it (nor is my code Haskell 98, because of multi-parameter classes), and all that seems left to try is Hat, whose Web site I can't reach at the moment.
Hi Konrad, The next release candidate of buddha 1.0 will work with ghc 6. I am in Sweden at the moment, which is VERY far away from home, so I won't be able to release it until next week. As for multi parameter type classes buddha might have enough unofficial support for them, but not functional dependencies at the moment. As a designer of debugging tools it is very useful for me to know what kind of things people want. Your input will help me to prioritise the things that I implement. If you have programs with bugs in them then I would love to see them. Test cases from "the wild world" are very useful to me also. I'm sorry that you are disappointed, but I hope that with your feedback the situation can be improved. Cheers, Bernie.
At 17:27 26/08/03 +0200, Konrad Hinsen wrote:
My Haskell experiments have reached a size in which debugging tools would be more than welcome, so I looked around, and was very disappointed. I tried Hood, which is a pain to use (lots of editing of the code required), I looked at Buddha but didn't want to downgrade to GHC 5 for trying it (nor is my code Haskell 98, because of multi-parameter classes), and all that seems left to try is Hat, whose Web site I can't reach at the moment.
So what are you Haskell programmers using for debugging in real life?
I have found that an approach based on unit testing (using HUnit) eliminates most requirements for a debugger. When I have problems, I resort to ad-hoc techniques, sometimes exposing (exporting) the inner workings of a module for testing/exploration purposes. #g ------------ Graham Klyne _________ GK@ninebynine.org ___|_o_o_o_|_¬ \____________/ (nb Helva) ~~~~~~~~~~~~~~~~~~ @Reading, River Thames http://www.ninebynine.org/Travels/2003Aug-Thames/Intro.html
Konrad Hinsen
... I didn't want to downgrade to GHC 5 for trying it (nor is my code Haskell 98, because of multi-parameter classes), and all that seems left to try is Hat, whose Web site I can't reach at the moment.
If the main Hat site is down: http://haskell.org/hat/ there is a mirror: http://www.cs.york.ac.uk/fp/hat/ Hat is also currently best used with ghc-5 rather than ghc-6, for fairly basic but tedious configuration issues, which we hope to clear up soon. However it does support hierarchical libraries, multi-parameter classes, and functional dependencies over and above plain Haskell'98.
So what are you Haskell programmers using for debugging in real life?
Hat :-) But I would say that.... Regards, Malcolm
On Tue, 26 Aug 2003, Konrad Hinsen wrote:
My Haskell experiments have reached a size in which debugging tools would be more than welcome, so I looked around, and was very disappointed. I tried Hood, which is a pain to use (lots of editing of the code required), I looked at Buddha but didn't want to downgrade to GHC 5 for trying it (nor is my code Haskell 98, because of multi-parameter classes), and all that seems left to try is Hat, whose Web site I can't reach at the moment.
So what are you Haskell programmers using for debugging in real life?
Konrad.
What are you trying to debug? I could write something that sounded more relevant if I knew. I haven't been doing anything that counts as real life programming, so my suggestions are probably biased towards trivial programs and low standards of correctness. In particular, I've never worried about space leaks. My advice is basically design suggestions and QuickCheck. Mostly I try to write my programs in small pieces and check those. It's not exactly debugging, but it makes debugging a lot easier. For trivial throwaway programs testing by hand in GHCI is enough (It's been enough for solving some old ACM problems. I love ReadP, and the monad transformers). QuickCheck gives a much better indication of correctness with much less manual labor. IIRC I used it when writing a unifier. I don't know how to test a GUI with QuickCheck, or custom monads (Browser, JVM, etc. custom=not in a typeclass, so you can't define a stub), but other than that I think it's the best thing short of a proof, and it has the advantage of being machine checked. Has anyone tried to write a system that would prove QuickCheck style properties from a function definition? The only tool I've used is HOOD. The version online needs a bit of hacking to work with GHC 6 (you need to resolve catch to Control.Exception.catch and fix some types). I like the output, and I didn't think the code changes were too bad. I had some trouble defining Observable instances when I was coding examples from "Recursion Schemes from Comonads", but I doubt your types are quite as convoluted (think DecoratingStructures at the type level). Do you want to avoid ANY code changes? Brandon
On Thursday 28 August 2003 07:05, Brandon Michael Moore wrote:
What are you trying to debug? I could write something that sounded more relevant if I knew.
The usual kind of problem is getting a wrong result, caused by a typo (misspelled constant, identifier, operator) or a not sufficiently general algorithm. What I would like to be able to do in such cases is trace the evaluation and look at intermediate results. Particular difficulties in Haskell: - Conditional tracing. Suppose a function is called 1000 times but I am interested in a particular intermediate result only when the third argument is greater then three. - Tracing a part of a value, say the first five elements of a list or the even-numbered elements of an array. In both cases the problem seems to be lazy evaluation. I can easily write down an expression for what I want to see, but unless its outcome is required in the computation, it will never be evaluated.
Mostly I try to write my programs in small pieces and check those. It's
Me too, of course. But it isn't always easy to test a function in isolation. If it takes complex data structures as input, then the only reasonable way to provide that input may be calling it from another piece of the code. Another problem is that I might not be able to verify the output of a particular function easily, it might take another one to post-process to something simpler or just something known.
QuickCheck gives a much better indication of correctness with much less manual labor. IIRC I used it when writing a unifier.
I haven't tried QuickCheck yet, but I have my doubts as to its suitability for numerical calculations, where you have to deal with precision and rounding issues.
The only tool I've used is HOOD. The version online needs a bit of hacking to work with GHC 6 (you need to resolve catch to Control.Exception.catch and fix some types). I like the output, and I didn't think the code changes were too bad. I had some trouble defining Observable instances
My problem was that most my code is polymorphic, not specifying concrete types but only type constraints. I found myself having to add "Observable a" to the type constraints of most of my functions just to be able to add a trace to one of them.
the type level). Do you want to avoid ANY code changes?
Not necessarily, but I'd prefer them to be local, at least restricted to one module. I ended up introducing (minor) bugs into rather unrelated code through typos made when adding "Observable" constraints. Konrad.
Konrad Hinsen
Particular difficulties in Haskell: - Conditional tracing. Suppose a function is called 1000 times but I am interested in a particular intermediate result only when the third argument is greater then three. - Tracing a part of a value, say the first five elements of a list or the even-numbered elements of an array.
The Hat solution is to trace everything, and then use a specialised query over the trace to narrow it to just the points you are interested in. At the moment, the hat-observe browser is the nearest to what you want - like HOOD, it permits you to see the arguments and results of a named function call, but additionally you can restrict the output to a named context, narrowed by normal Haskell-style patterns for the arguments and results. e.g. foo True [2,3,_] = Branch _ (Leaf _) in g asks for all applications of foo called from g, where foo's first argument is True, its second argument is a list of three elements beginning with 2 and 3, and the result of the call is a tree with a leaf on the immediate right of the top-most branch. Over the next year, we hope to have a research student looking at extending the query language of hat-observe. For instance, I can imagine adding Haskell-style guards, which would make it possible to express your first example as foo _ _ x | x > 3 -- third argument must be greater than three Another idea is to permit real Haskell expressions to post-process the result of the trace query, rather like meta-programming. So your second example might look something like take 5 [| foo 3 |] -- show only the first five elements of the result list evens [| mkarray _ |] -- select and print only the even-indexed array elements where evens arr = show (map (\i->arr!i) [b, succ(succ(b)) .. t]) where b = fst (bounds arr) t = snd (bounds arr)
If it takes complex data structures as input, then the only reasonable way to provide that input may be calling it from another piece of the code.
Exactly, and this is where Hat is most useful, because it traces the real run of the program, recording all the intermediate data structures.
QuickCheck gives a much better indication of correctness with much less manual labor.
I can recommend QuickCheck as well, although Koen or John may wish to comment on how suitable they think it would be for numerical work. QuickCheck and Hat can be made to work together nicely. There is a version of QuickCheck in development which works by first running the ordinary program with lots of random test data. If a failure is found, it prunes the test case to the minimal failing case, and passes that minimal case to a Hat-enabled version of the program, which can then be used to investigate the cause of the failure.
the type level). Do you want to avoid ANY code changes?
Not necessarily, but I'd prefer them to be local, at least restricted to one module. I ended up introducing (minor) bugs into rather unrelated code through typos made when adding "Observable" constraints.
The need to alter the program under test is the biggest disadvantage of HOOD in my opinion since, as you have shown, it can lead to new bugs. Hat largely avoids this pitfall. I believe there is also a version of HOOD more fully integrated with Hugs, which removes the need to derive Observable instances and add extra Observable constraints on functions. Regards, Malcolm
On Thursday 28 August 2003 12:57, Malcolm Wallace wrote:
The Hat solution is to trace everything, and then use a specialised query over the trace to narrow it to just the points you are interested
That sounds risky with programs that treat megabytes of data. It isn't always possible to test with small data sets, e.g. when different algorithms are used depending on the size of the problem.
in. At the moment, the hat-observe browser is the nearest to what you want - like HOOD, it permits you to see the arguments and results of a named function call, but additionally you can restrict the output
I am mostly interested in intermediate values.
Another idea is to permit real Haskell expressions to post-process the result of the trace query, rather like meta-programming. So your second
That sounds like a very flexible approach. Could one perhaps do this *while* the trace is being constructed, in a lazy evaluation fashion, such that unwanted trace data is never generated and stored?
QuickCheck and Hat can be made to work together nicely. There is a version of QuickCheck in development which works by first running the ordinary program with lots of random test data. If a failure is found, it prunes the test case to the minimal failing case, and passes that minimal case to a Hat-enabled version of the program, which can then be used to investigate the cause of the failure.
That sounds very useful. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais -------------------------------------------------------------------------------
Konrad Hinsen
The Hat solution is to trace everything, and then use a specialised query over the trace to narrow it to just the points you are interested
That sounds risky with programs that treat megabytes of data. It isn't always possible to test with small data sets, e.g. when different algorithms are used depending on the size of the problem.
Indeed, Hat can chew up disk space (to store the trace) at a frightening rate. The fastest I have witnessed so far is 50Mb per second on one particularly intensive run, and we have certainly bumped into a 2Gb single-file limit once or twice as well. But backing store is cheap, you can interrupt a computation early to use the partial trace, and you can always 'trust' parts of the computation to reduce the amount of storage needed. However, you probably don't know ahead of time where the fault is, so our philosophy is that it is better to expensively store (once) everything that might turn out to be relevant, and then cheaply browse it, than to go through several cycles of edit/compile/expensive-run.
like HOOD, it permits you to see the arguments and results of a named function call,
I am mostly interested in intermediate values.
Intermediate values are always the result of an expression, and this is exactly what hat-observe shows you - expressions and their results. There is another trace-browser, hat-trail, used for tracing backwards from an error or faulty output value to the expressions that produced it. This likewise shows all the intermediate values along the way.
Another idea is to permit real Haskell expressions to post-process the result of the trace query, rather like meta-programming. So your second
That sounds like a very flexible approach. Could one perhaps do this *while* the trace is being constructed, in a lazy evaluation fashion, such that unwanted trace data is never generated and stored?
What an interesting idea. Yes, I imagine that might be another good avenue for us to explore. Provided one could express the desired trace query *before* running the program, at runtime the system would need only to construct that part of the trace which is relevant to the query. This could improve the efficiency of tracing, but on the other hand it severely limits the amount of information available at the end. Regards, Malcolm
On Thursday 28 August 2003 17:07, Malcolm Wallace wrote:
Indeed, Hat can chew up disk space (to store the trace) at a frightening rate. The fastest I have witnessed so far is 50Mb per second on one particularly intensive run, and we have certainly bumped into a 2Gb single-file limit once or twice as well. But backing
I do calculations that hit the 2GB limit for the file holding the intended results... (not in Haskell for the moment) I don't want to think about the number of disks I'd have to buy to hold a trace of such a run.
partial trace, and you can always 'trust' parts of the computation to reduce the amount of storage needed. However, you probably don't
Ah, OK, that should help.
like HOOD, it permits you to see the arguments and results of a named function call,
I am mostly interested in intermediate values.
Intermediate values are always the result of an expression, and this is exactly what hat-observe shows you - expressions and their results.
But if I can only see the results of named function calls, I don't get the intermediate values. A typical situation would be a function that looks like f x = sum [...] When my unit test reports that the sum is wrong, I'd then like to use a debugging tool to look at the list before summation.
There is another trace-browser, hat-trail, used for tracing backwards from an error or faulty output value to the expressions that produced it. This likewise shows all the intermediate values along the way.
That looks very useful indeed. Konrad -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais -------------------------------------------------------------------------------
Konrad Hinsen
But if I can only see the results of named function calls, I don't get the intermediate values.
A typical situation would be a function that looks like
f x = sum [...]
When my unit test reports that the sum is wrong, I'd then like to use a debugging tool to look at the list before summation.
... so in hat-observe, you could ask to see "sum _" only in the context of a call from f, which might report something like sum [1,2,3] = 6 sum [1,2,4] = 9 and you could then use hat-trail to examine why the second equation gave the wrong answer, by looking into its internals. Or on the other hand, the addition may be correct but the fault is that the wrong list argument was passed to "sum" in the first place, in which case you explore where that came from instead. Regards, Malcolm
On Thu, Aug 28, 2003 at 11:33:40AM +0200, Konrad Hinsen wrote:
I haven't tried QuickCheck yet, but I have my doubts as to its suitability for numerical calculations, where you have to deal with precision and rounding issues.
I'd still recommend trying it. In the worst case, you can just specify a range of acceptable answers. e.g. prop_minus_plus a b = abs(((a-b)+b) - a) < abs(b)*1.0e-15 would make sure that if you subtract b from a and then add it you get back a again to a reasonable precision. But in cases where it is possible I strongly recommend using equality tests. I'm right now working (at work) on parallelizing a numerical code (in C++, not haskell, but the issues are similar), and I've managed to assure that most the results are identical regardless of how the data is split up, which is a very powerful check. For things like summations, of course, the answer will depend on the number of processors, but it was definitely worth the effort (a day) to make all the intermediate results are calculated in precisely the same order (that is, order of additions) regardless of number of processors, just so I could have the strict equality test. -- David Roundy http://www.abridgegame.org
participants (7)
-
Bernard James POPE -
Brandon Michael Moore -
David Roundy -
David Roundy -
Graham Klyne -
Konrad Hinsen -
Malcolm Wallace