testing and the culture of Haskell
Folks, A friend asked me about Test-Driven Development (TDD) within the FP community. I realize the FP community is larger than a Haskell mailing list, but I wanted to get a sense from this camp. I'm a fan of Haskell but I can't say that I know the culture yet. That said, I'm writing as *a fan* and not a critic. These are genuine, earnest questions. Q: Is TDD advocated the Haskell community? Is it controversial? Is it even on the radar? Q: If TDD is not advocated: why? Is there something about the Zen of Haskell development that is an impedance mismatch with TDD? Q: Note that TDD and "writing tests" are different things. With respect to "writing tests", I know that HUnit exists and that RWH has a chapter on quality assurance. Given that, I'd like to know: how widely is HUnit used? If you were to start a new Haskell project, would you include HUnit (a) immediately (b) eventually (c) maybe (d) another adjective ? sincerely Michael Easter -- ---------------------- Michael Easter http://codetojoy.blogspot.com -> Putting the thrill back in blog http://twitter.com/patentlyfalse -> Satirical tech headlines (and nothing but) http://youtube.com/ocitv -> Fun people doing serious software engineering
Michael Easter wrote:
Q: Is TDD advocated the Haskell community? Is it controversial? Is it even on the radar?
we're not big on hyped, capitalized words, but many of us like tests. There's even a tool, "HPC" (Haskell Program Coverage) that you can use when you run your tests to see how much of the code is reached at all by the tests. Actually, in terms of writing tests before writing code (which I sometimes do)... what Haskellers often do even more often is to write type-signature for functions before they write the function's code. This helps testing (i.e. test failure = compile-time type errors) as well as thinking about what you want the function to do. Types are generally more useful-for-thinking in FP than in imperative languages
Q: Note that TDD and "writing tests" are different things. With respect to "writing tests", I know that HUnit exists and that RWH has a chapter on quality assurance.
I would use QuickCheck first... have you taken a look at it? -Isaac
Hello Michael How about - Type Driven Development? Personally, I like to work out the type a function should have, then set about writing it. So I usually start from the type signature and stub out the right-hand side of the definition (i.e. all the function after the equals sign) with 'undefined'. Working like this is far from universal in functional programming - it would be unnatural in Scheme for instance, though some people using PLT Scheme appear to start development by writing contract signatures. It certainly isn't universal in Haskell either - some people like to write their functions then fill in the type signature afterwards (if you do it this way, providing your function compiles, you can query its 'most general' type at GHCi's prompt). ML used to have the proviso that type checking was always guaranteed to terminate so type signatures were discretionary, I'm not sure if this is still the case with O'Caml as it now seems to have quite an extended type system. Certain bits of Haskell need explicit type signatures - one example is polymorphic recursion. I suspect very few people programming in Haskell use tests as the primary 'driver' of their program designs (my interpretation of Test Driven Developments slogans is that testing very much drives design). I'd guess quite a number of people develop 'in tandem' with QuickCheck (rather than strictly 'first' with QuickCheck) - for example, there is a blog post on the XMonad window system covering how the authors worked in this way. http://cgi.cse.unsw.edu.au/~dons/blog/2007/05/01 Best wishes Stephen
I'm a fan of Haskell but I can't say that I know the culture yet.
PS. I'd like to think Haskell has at least 'cultures' (or schools, factions?) rather than a unified culture or way. If you like developing test-first, why change if it works.
Michael,
A friend asked me about Test-Driven Development (TDD) within the FP community.
My brother is a TDD (Test-Driven Development) guru but not a Haskeller, and I'm a Haskeller, and I've thought a lot about this question and asked around in the community. My brother and I are always attempting to brainwash each other. It's been quite effective so I essentially agree with TDD. Here's my 2c worth on your questions:
Q: Is TDD advocated the Haskell community? Is it controversial? Is it even on the radar?
I've never heard of anyone advocating TDD in the Haskell community. It isn't controversial - it's on the radar for a minority, presumably in proportion to how much of their heads are in the mainstream community vs. the functional community. So the functional community doesn't generally talk about TDD.
Q: If TDD is not advocated: why? Is there something about the Zen of Haskell development that is an impedance mismatch with TDD?
After much pondering, and with the proviso that everything is In My Humble Opinion, I believe there are some very sound technical reason why TDD doesn't quite work for Haskell. TDD is really intended to catch two kinds of errors: Programming mistakes, and wrong logic. TDD proponents say "test logic only", but as a happy side effect they're catching a lot of programming mistakes too, and this adds significantly to the value of TDD. Another reason behind TDD is that if your code is testable, then it forces it to be detachable from its "environment". This makes the code more composable, but it goes much further than that: The TDD process actually drives the design, and, one step further still - leads to an "emergent" design method. So, adding a fourth point, the summary is that TDD is for 1. verifying logic; 2. detecting programming mistakes; 3. promoting a clean, and even emergent design; 4. making it so you can re-factor without breaking your code. (apologies to TDD people if I misunderstand something) Now, Haskell is *very* good at giving you 2, 3 and 4. I have no idea whether Haskell gives 2, 3 and 4 better or worse than C# or Java coded in a TDD style. Certainly it gives it to you with a whole lot less typing. Haskell also contributes to 1, in that the type system *can be used* to significantly tie down your logic, and make mistakes less likely. But ultimately Haskell can't make you write your logic correctly. I am saying that I think Haskell gives you about 70-90% of what TDD gives you, but with less fuss. The remaining 10-30% you still need testing for. The key point, is that in functional programming, TDD is not really needed to "drive the design", because the language itself does the same job, but in a slightly different way: Good design is the path of least resistance in Haskell, and since re-factoring is so safe and easy, it almost automatically tends in that direction. So I think the main purpose of TDD ("driving the design") doesn't apply in Haskell. The only part of TDD that applies to Haskell is verifying logic, but this can be done after the fact, with the added risk (not shared by TDD) that you won't bother. The way Haskell programmers usually work, which I think is the best practice for the language, is to drive the design just through plain Haskell coding, but to have enough self discipline to recognize where your logic needs to be verified, and write tests in those cases. (QuickCheck makes it so easy!) I would certainly urge functional programmers to grasp the (actually somewhat subtle) philosophy behind TDD.
Q: Note that TDD and "writing tests" are different things. With respect to "writing tests", I know that HUnit exists and that RWH has a chapter on quality assurance.
Given that, I'd like to know: how widely is HUnit used? If you were to start a new Haskell project, would you include HUnit (a) immediately (b) eventually (c) maybe (d) another adjective ?
I personally use HUnit a lot, so I use test-framework, which gives both QuickCheck and HUnit. I think the best way is to use QuickCheck if you possibly can, and if not, use HUnit (either for IO-bound code, or for situations where it's easier to specify the test cases yourself than to induce QuickCheck to generate them). I do about 1/5th of my tests in HUnit, so I would always anticipate that I would want it, so my answer is (a). Steve Michael Easter wrote:
Folks,
A friend asked me about Test-Driven Development (TDD) within the FP community. I realize the FP community is larger than a Haskell mailing list, but I wanted to get a sense from this camp.
I'm a fan of Haskell but I can't say that I know the culture yet. That said, I'm writing as *a fan* and not a critic. These are genuine, earnest questions.
Q: Is TDD advocated the Haskell community? Is it controversial? Is it even on the radar?
Q: If TDD is not advocated: why? Is there something about the Zen of Haskell development that is an impedance mismatch with TDD?
Q: Note that TDD and "writing tests" are different things. With respect to "writing tests", I know that HUnit exists and that RWH has a chapter on quality assurance.
Given that, I'd like to know: how widely is HUnit used? If you were to start a new Haskell project, would you include HUnit (a) immediately (b) eventually (c) maybe (d) another adjective ?
sincerely Michael Easter
-- ---------------------- Michael Easter http://codetojoy.blogspot.com -> Putting the thrill back in blog
http://twitter.com/patentlyfalse -> Satirical tech headlines (and nothing but)
http://youtube.com/ocitv -> Fun people doing serious software engineering
------------------------------------------------------------------------
_______________________________________________ Beginners mailing list Beginners@haskell.org http://www.haskell.org/mailman/listinfo/beginners
Michael Easter wrote:
Q: If TDD is not advocated: why? Is there something about the Zen of Haskell development that is an impedance mismatch with TDD?
Static typing is a great help, it is not uncommon for Haskell code to be correct on the first try once you convinced GHC that it's type correct. There is much less need for TDD than in other languages. Moreover, programming with pure functions offers a whole new approach to program design. For instance, you can start with an inefficient but obviously correct program and use laws and equations to rewrite that to an efficient one. (You could call that "Proof Driven Design"). Here a prime example: Richard Bird. A program to solve Sudoku. http://www.cs.tufts.edu/~nr/comp150fp/archive/richard-bird/sudoku.pdf
Q: Note that TDD and "writing tests" are different things. With respect to "writing tests", I know that HUnit exists and that RWH has a chapter on quality assurance.
The purpose of HUnit seems to be testing code that mainly involves IO. For the purely functional part, people usually use QuickCheck because you don't have to think about test cases, it will generate them automatically! Regards, Heinrich Apfelmus -- http://apfelmus.nfshost.com
2010/1/20 Heinrich Apfelmus
Static typing is a great help, it is not uncommon for Haskell code to be correct on the first try once you convinced GHC that it's type correct. There is much less need for TDD than in other languages.
Ah, maybe the type checker is given all credit here, when perhaps it should be shared? With functional programming languages you are largely programming with expressions - Scheme has the (begin ... ...) form for sequencing and ML has sequence control structure (;), but there is simply less control flow in typical functional programs than imperative ones (there is also less use of assignment but that's hardly news of course). Figuratively speaking, functional programs have 'fewer movable parts' to go wrong (vis-a-vis incrementing variable for loop indexes etc. whereas map, fold, unfold, ... can be written once and used anywhere), hence static-typing + control-flow reduction (+ limited use of side effects) hopefully leads to first-time correctness. Best wishes Stephen
Stephen Tetley wrote:
Heinrich Apfelmus wrote:
Static typing is a great help, it is not uncommon for Haskell code to be correct on the first try once you convinced GHC that it's type correct. There is much less need for TDD than in other languages.
Ah, maybe the type checker is given all credit here, when perhaps it should be shared?
With functional programming languages you are largely programming with expressions - Scheme has the (begin ... ...) form for sequencing and ML has sequence control structure (;), but there is simply less control flow in typical functional programs than imperative ones (there is also less use of assignment but that's hardly news of course).
Figuratively speaking, functional programs have 'fewer movable parts' to go wrong (vis-a-vis incrementing variable for loop indexes etc. whereas map, fold, unfold, ... can be written once and used anywhere), hence static-typing + control-flow reduction (+ limited use of side effects) hopefully leads to first-time correctness.
Fair enough. However, in a sense, one can interpret the pure in purely functional as a property of the type constructor (->) , i.e. a function of type (a -> b) is guaranteed to not have side effects. In this light, ML and Scheme are lacking a very important type: they only have functions with side effects. Or more precisely, their type system does not distinguish between functions with and without side effects. Regards, Heinrich Apfelmus -- http://apfelmus.nfshost.com
Heinrich Apfelmus wrote:
However, in a sense, one can interpret the pure in purely functional as a property of the type constructor (->) , i.e. a function of type (a -> b) is guaranteed to not have side effects. In this light, ML and Scheme are lacking a very important type: they only have functions with side effects.
Or more precisely, their type system does not distinguish between functions with and without side effects.
that's not actually more precise! Haskell doesn't entirely do that either. consider: f :: Int -> IO Int f x = return (x + 1) No side effects! But other ->IO typed values do yield side-effects. It is exactly as if ML and Scheme contain only the type of (->) composed with IO. (Well, actually Scheme is dynamically typed so it's a bit silly to say so..) (and Scheme allows functions with zero arguments, which are different from its non-side-effecting values.. not sure about ML) -Isaac
On Wed, 2010-01-20 at 18:13 -0500, Isaac Dupree wrote:
Heinrich Apfelmus wrote:
However, in a sense, one can interpret the pure in purely functional as a property of the type constructor (->) , i.e. a function of type (a -> b) is guaranteed to not have side effects. In this light, ML and Scheme are lacking a very important type: they only have functions with side effects.
Or more precisely, their type system does not distinguish between functions with and without side effects.
that's not actually more precise! Haskell doesn't entirely do that either. consider: f :: Int -> IO Int f x = return (x + 1) No side effects! But other ->IO typed values do yield side-effects.
It is exactly as if ML and Scheme contain only the type of (->) composed with IO. (Well, actually Scheme is dynamically typed so it's a bit silly to say so..) (and Scheme allows functions with zero arguments, which are different from its non-side-effecting values.. not sure about ML)
-Isaac
Hmm. Is it possible to distinguish them? Consider: f :: Integer -> IO Integer f x = if cond then putStrLn "Hello World!" >> return x else return (x + 1) where cond = ... As far as I remember it is impossible to determine in general if cond will ever be true[1]. Hence it is unknown in general if f is pure or have side-effects. Generally IO marks that function have side effects so there should be no point in testing if it has none (although it may be worth to check if only the specific side effects occurred). So it rather (forgetting about unsafe* functions): whenever v is not a IO it has no effects[2]. Regards [1] However as Int is bounded it is possible for it [2] I am not sure how correct it is but I think about a -> IO b as function from a to some sub-program returning b with haskell program being combination of programs rather then a unpure function from a to b.
Isaac, Isaac Dupree wrote:
Heinrich Apfelmus wrote:
Or more precisely, their type system does not distinguish between functions with and without side effects.
that's not actually more precise! Haskell doesn't entirely do that either. consider: f :: Int -> IO Int f x = return (x + 1) No side effects! But other ->IO typed values do yield side-effects.
All Haskell functions are pure without exception. For example: greet :: String -> IO () greet name = putStrLn $ "Hello, "++name This is a pure function from String to IO (). This function (like all Haskell functions) has no side effects. Its return value of type IO () merely _represents_ an IO action. The runtime system knows how to act on this representation. This also means that there is no such thing in Haskell as marking a function as side-effecting. This distinction may be subtle, but it's important. Steve
On Thu, Jan 21, 2010 at 11:20 AM, Stephen Blackheath [to
Haskell-Beginners]
All Haskell functions are pure without exception. For example:
greet :: String -> IO () greet name = putStrLn $ "Hello, "++name
This is a pure function from String to IO (). This function (like all Haskell functions) has no side effects. Its return value of type IO () merely _represents_ an IO action. The runtime system knows how to act on this representation.
This also means that there is no such thing in Haskell as marking a function as side-effecting.
This distinction may be subtle, but it's important.
Coming from an imperative background, I have found that distinction to be confusing. I like to understand how things work. For example, haskell's lazyness confused me a lot until I heard about thunks. Back to IO. What exactly would be the representation of an IO action, if not an abstract notion ? Hopefully that is optimised out by the compiler. Indeed, If I look at the compiled output of a simple program, it looks to me like the effects are executed within the function, and no special structure is returned. David.
David Virebayre wrote:
Stephen Blackheath wrote:
All Haskell functions are pure without exception. For example:
Back to IO. What exactly would be the representation of an IO action, if not an abstract notion ? Hopefully that is optimised out by the compiler. Indeed, If I look at the compiled output of a simple program, it looks to me like the effects are executed within the function, and no special structure is returned.
Sure, it's an abstraction, which means that the compiler is free to optimize it ruthlessly. But you are also free to implement it differently, like for the purpose of testing: Swierstra und Altenkirch. Beauty in the beast. http://www.cse.chalmers.se/~wouter/Publications/BeautyInTheBeast.pdf Regards, Heinrich Apfelmus -- http://apfelmus.nfshost.com
Isaac Dupree wrote:
Heinrich Apfelmus wrote:
However, in a sense, one can interpret the pure in purely functional as a property of the type constructor (->) , i.e. a function of type (a -> b) is guaranteed to not have side effects. In this light, ML and Scheme are lacking a very important type: they only have functions with side effects.
Or more precisely, their type system does not distinguish between functions with and without side effects.
that's not actually more precise! Haskell doesn't entirely do that either. consider: f :: Int -> IO Int f x = return (x + 1) No side effects! But other ->IO typed values do yield side-effects.
It is exactly as if ML and Scheme contain only the type of (->) composed with IO. (Well, actually Scheme is dynamically typed so it's a bit silly to say so..) (and Scheme allows functions with zero arguments, which are different from its non-side-effecting values.. not sure about ML)
Yes, "distinguish" in the sense of "optional distinction by annotation" instead of "automatically detect whether a function has side effects or not". (By the way, in a sense it is possible to detect that f has no side effects. Namely, it's most general type is polymorphic in the monad: f :: Monad m => Int -> m Int ) This is much like Java lacking (having lacked?) types whose values cannot be null , compared to a vs Maybe a in Haskell. Regards, Heinrich Apfelmus -- http://apfelmus.nfshost.com
I'm pretty new to Haskell, so take what I say with a very large grain of salt. But this is a topic I've been thinking about because I happen to be teaching a lab on TDD to Java and C developers at the moment, while I myself am programming in Haskell in my research. So perhaps my impressions will be of some use to you. There are two reasons why I'm less likely to do TDD in Haskell. By far the most important is the availability of a tool like QuickCheck. QuickCheck changes everything. I don't write tests, I write properties. QuickCheck generates the tests on the fly, and does a better job than I could. There's a big difference between writing properties and writing tests. If I were going to develop a large application in Haskell, I'd try to honour the spirit of TDD by defining properties before implementing functions. A secondary reason is the availability of the interpreter. I test functions in the interpreter as I write them. Of course, I'm not doing a very thorough job, but I'll use QuickCheck to do the "real" testing. I use the interpreter to test whether the function does what I designed it to do, with some typical inputs. I then use QuickCheck to verify that the function behaves nicely no matter what inputs it gets, including inputs I might not have even thought of trying. FWIW, I do plan to use HUnit to test the non-functional part of my application.
On Jan 20, 2010, at 05:31 , Amy de Buitléir wrote:
properties and writing tests. If I were going to develop a large application in Haskell, I'd try to honour the spirit of TDD by defining properties before implementing functions.
This is a lot like "programming by contract", btw. (Just something to think about.) It's also where you can best take advantage of static typing, by implementing the properties as types so the type system helps you guarantee correctness. -- brandon s. allbery [solaris,freebsd,perl,pugs,haskell] allbery@kf8nh.com system administrator [openafs,heimdal,too many hats] allbery@ece.cmu.edu electrical and computer engineering, carnegie mellon university KF8NH
Michael Easter 쓴 글:
Q: Note that TDD and "writing tests" are different things. With respect to "writing tests", I know that HUnit exists and that RWH has a chapter on quality assurance.
Given that, I'd like to know: how widely is HUnit used? If you were to start a new Haskell project, would you include HUnit (a) immediately (b) eventually (c) maybe (d) another adjective ?
Don't forget about QuickCheck which allows us property based automatically generated random testing. With QuickCheck, one can practice much more powerful form of TDD, and this is a already a common practice in Haskell development. QuickCheck works well with pure functions, and HUnit is more for the IO related actions.
participants (10)
-
Ahn, Ki Yung -
Amy de Buitléir -
Brandon S. Allbery KF8NH -
David Virebayre -
Heinrich Apfelmus -
Isaac Dupree -
Maciej Piechotka -
Michael Easter -
Stephen Blackheath [to Haskell-Beginners] -
Stephen Tetley