Haskell and the Software design process

Hi I was just wondering what methods are best to design/model the software in bigger projects when you are planning to use Haskell. Is there no difference compared to other languages? Are there any Haskell tools? Cheers, Jaco PS. "Software design in Haskell" didn't give relevant hits. :(

jaco.van.iterson:
Hi
I was just wondering what methods are best to design/model the software in bigger projects when you are planning to use Haskell. Is there no difference compared to other languages? Are there any Haskell tools?
I don't believe anyone has written a "Programming Haskell in the Large" book (or any other similar functional language??), but there is lots of experience in this community working on big, long lived code bases. Some key points: * reduce the interactions between components by relying on pure interfaces * use types to encode the design into a machine checkable form * use QuickCheck and HPC to ensure property driven development and good coverage * use Hackage packages * use purely functional data structures to model key parts of the world you're talking to. Model-driven design with easy to verify logic * picking a good data type (like a zipper) will make hundreds of unit tests meaningless -- improving productivity. * ensure components have 'axiomatic' interfaces -- reduce complexity by avoiding redundancy * if at all possible ensure your core algorithms and logic are referentially transparent -- this will dramatically simplify maintainanace and QA effort * Use GHC -Wall, QuickCheck , -fhpc, and retainer profiling to ensure space invariants. * Run a set of QC properties on every commit. * Avoid partial functions * Avoid side effecting code * Every bug should be mirrored as a property or type assertion.

On Sun, May 2, 2010 at 4:10 PM, Don Stewart
I don't believe anyone has written a "Programming Haskell in the Large" book (or any other similar functional language??), but there is lots of experience in this community working on big, long lived code bases.
Some key points: [...] * picking a good data type (like a zipper) will make hundreds of unit tests meaningless -- improving productivity.
Don, What sort of tests were you thinking of that a zipper would render pointless? Could you elaborate? Thanks, Brad

brad.larsen:
On Sun, May 2, 2010 at 4:10 PM, Don Stewart
wrote: I don't believe anyone has written a "Programming Haskell in the Large" book (or any other similar functional language??), but there is lots of experience in this community working on big, long lived code bases.
Some key points: [...] * picking a good data type (like a zipper) will make hundreds of unit tests meaningless -- improving productivity.
Don,
What sort of tests were you thinking of that a zipper would render pointless? Could you elaborate?
Well, specifically, zippers statically ensure that you always have a valid index into a set. No need for arr ! 10 out of bounds checks, when the type guarantees that indexing can never be out of bounds. Clever types move invariants from dynamic properties to ones the compiler can prove, saving you effort. -- Don

On Sun, May 2, 2010 at 9:18 PM, Edgar Z. Alvarenga
On Sun, 02/May/2010 at 13:10 -0700, Don Stewart wrote:
* Avoid partial functions
Why?
Edgar
Ever place you use a partial function, you need to verify that its usage is in fact safe. Otherwise, you risk pattern match failures, undefined, nontermination, and other types of nasties. If you can structure your code so none of your functions are partial, verification that their usage is safe is a whole lot easier. :-) Brad

I'm a little confused about this too. I've seen many functions defined like:
f x = (\s -> ...)
which is a partial function because it returns a function and is the same as:
f x s = ...
Off the top of my head the State monad makes extensive use if this
style. Is this bad?
- deech
On 5/2/10, Bradford Larsen
On Sun, May 2, 2010 at 9:18 PM, Edgar Z. Alvarenga
wrote: On Sun, 02/May/2010 at 13:10 -0700, Don Stewart wrote:
* Avoid partial functions
Why?
Edgar
Ever place you use a partial function, you need to verify that its usage is in fact safe. Otherwise, you risk pattern match failures, undefined, nontermination, and other types of nasties.
If you can structure your code so none of your functions are partial, verification that their usage is safe is a whole lot easier. :-)
Brad _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

On 3 May 2010 14:17, aditya siram
I'm a little confused about this too. I've seen many functions defined like: f x = (\s -> ...) which is a partial function because it returns a function and is the same as: f x s = ...
No, that's a partially applied function. A partial function is one such as: secondElement (_:x:_) = x Note that it is only defined for lists with at least two elements, for any other list (i.e. singleton or empty) it will throw an error; -fwarn-incomplete-patterns (which is included in -Wall) tells you about these. You can also argue that functions such as head are partial, as they explicitly throw an error if the input data isn't correct. Partial functions are bad because if you accidentally use one the wrong way, your entire program crashes in a flaming wreck. It's much better to do something like this: safeSecondElement (_:x:_) = Just x safeSecondElement _ = Nothing This will work with all possible input types. For more information, see http://en.wikipedia.org/wiki/Partial_function (from the mathematical perspective). -- Ivan Lazar Miljenovic Ivan.Miljenovic@gmail.com IvanMiljenovic.wordpress.com

Cool. That makes way more sense.
I thought that ghc -Wall picked these up. So at least this problem
would go away if warnings were turned on (and heeded) by default.
Besides that from my own experience I'd strongly encourage people to
use HLint [1] .
-deech
[1 ] http://hackage.haskell.org/package/hlint
On 5/3/10, Ivan Miljenovic
On 3 May 2010 14:17, aditya siram
wrote: I'm a little confused about this too. I've seen many functions defined like: f x = (\s -> ...) which is a partial function because it returns a function and is the same as: f x s = ...
No, that's a partially applied function.
A partial function is one such as:
secondElement (_:x:_) = x
Note that it is only defined for lists with at least two elements, for any other list (i.e. singleton or empty) it will throw an error; -fwarn-incomplete-patterns (which is included in -Wall) tells you about these.
You can also argue that functions such as head are partial, as they explicitly throw an error if the input data isn't correct.
Partial functions are bad because if you accidentally use one the wrong way, your entire program crashes in a flaming wreck. It's much better to do something like this:
safeSecondElement (_:x:_) = Just x safeSecondElement _ = Nothing
This will work with all possible input types.
For more information, see http://en.wikipedia.org/wiki/Partial_function (from the mathematical perspective).
-- Ivan Lazar Miljenovic Ivan.Miljenovic@gmail.com IvanMiljenovic.wordpress.com

I missed the part where you talked about -Wall. Sorry.
On 5/3/10, aditya siram
Cool. That makes way more sense.
I thought that ghc -Wall picked these up. So at least this problem would go away if warnings were turned on (and heeded) by default.
Besides that from my own experience I'd strongly encourage people to use HLint [1] .
-deech
[1 ] http://hackage.haskell.org/package/hlint
On 5/3/10, Ivan Miljenovic
wrote: On 3 May 2010 14:17, aditya siram
wrote: I'm a little confused about this too. I've seen many functions defined like: f x = (\s -> ...) which is a partial function because it returns a function and is the same as: f x s = ...
No, that's a partially applied function.
A partial function is one such as:
secondElement (_:x:_) = x
Note that it is only defined for lists with at least two elements, for any other list (i.e. singleton or empty) it will throw an error; -fwarn-incomplete-patterns (which is included in -Wall) tells you about these.
You can also argue that functions such as head are partial, as they explicitly throw an error if the input data isn't correct.
Partial functions are bad because if you accidentally use one the wrong way, your entire program crashes in a flaming wreck. It's much better to do something like this:
safeSecondElement (_:x:_) = Just x safeSecondElement _ = Nothing
This will work with all possible input types.
For more information, see http://en.wikipedia.org/wiki/Partial_function (from the mathematical perspective).
-- Ivan Lazar Miljenovic Ivan.Miljenovic@gmail.com IvanMiljenovic.wordpress.com

On Sun, May 2, 2010 at 9:24 PM, Ivan Miljenovic
On 3 May 2010 14:17, aditya siram
wrote: I'm a little confused about this too. I've seen many functions defined like: f x = (\s -> ...) which is a partial function because it returns a function and is the same as: f x s = ...
No, that's a partially applied function.
A partial function is one such as:
secondElement (_:x:_) = x
Note that it is only defined for lists with at least two elements, for any other list (i.e. singleton or empty) it will throw an error; -fwarn-incomplete-patterns (which is included in -Wall) tells you about these.
You can also argue that functions such as head are partial, as they explicitly throw an error if the input data isn't correct.
Partial functions are bad because if you accidentally use one the wrong way, your entire program crashes in a flaming wreck. It's much better to do something like this:
safeSecondElement (_:x:_) = Just x safeSecondElement _ = Nothing
This will work with all possible input types.
For more information, see http://en.wikipedia.org/wiki/Partial_function (from the mathematical perspective).
-- Ivan Lazar Miljenovic Ivan.Miljenovic@gmail.com IvanMiljenovic.wordpress.com _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Of course, there are situations where it is really awkward to not use partial functions, basically because you *know* that an invariant is satisfied and there is no sane course of action if it isn't. To take a contrived example: f ys = let xs = (1:ys) in last xs uses the partial function "last". Rewriting it in the "non-partial style" gives f ys = case (1:ys) of [] -> Nothing xs -> Just (last xs) but what possible meaning could a caller of the function ascribe to a "Nothing" result? It just means that there is a bug in f, which is what an error would tell you anyway. Of course, this particular function could easily be rewritten in such a way to be total, but I believe there are more complex examples where it is awkward/impossible to do so. Alex

On 3 May 2010 14:35, Alexander Dunlap
Of course, there are situations where it is really awkward to not use partial functions, basically because you *know* that an invariant is satisfied and there is no sane course of action if it isn't.
True, like "map head . group". -- Ivan Lazar Miljenovic Ivan.Miljenovic@gmail.com IvanMiljenovic.wordpress.com

On May 3, 2010, at 6:35 AM, Alexander Dunlap wrote:
Of course, there are situations where it is really awkward to not use partial functions, basically because you *know* that an invariant is satisfied
That falls under Don's: "Use types to encode the design into a machine checkable form" Or as Yaron Minsky put it: "Make illegal states unrepresentable" When you do this, your functions don't need to be partial, if you know that it doesn't matter. Sebastian -- Underestimating the novelty of the future is a time-honored tradition. (D.G.)

* On Sunday, May 02 2010, Alexander Dunlap wrote:
Of course, there are situations where it is really awkward to not use partial functions, basically because you *know* that an invariant is satisfied and there is no sane course of action if it isn't. To take a contrived example:
f ys = let xs = (1:ys) in last xs
uses the partial function "last". Rewriting it in the "non-partial style" gives
f ys = case (1:ys) of [] -> Nothing xs -> Just (last xs)
but what possible meaning could a caller of the function ascribe to a "Nothing" result? It just means that there is a bug in f, which is what an error would tell you anyway. Of course, this particular function could easily be rewritten in such a way to be total, but I believe there are more complex examples where it is awkward/impossible to do so.
You're just showing how difficult it is to reason about when the use of partial functions is safe: undefined), with your `non-partial style' version, you still leave open the case of (Just undefined), since we cannot convince ourselves that last only fails on empty lists here... One case where Maybe or Either might be less preferable to an exception is for the output of a lexer (ex. alex does this) because it means you can get some tokens without without forcing the whole input, and do some IO based on the results even if there is a mistake in either reading the input from disk or lexing it. But I think this situation of (ab)using lazy IO and exceptions is an oversimplification, and you can find appropriate alternatives such as iteratee which are more explicit about what is going on. -- Adam

On Sun, 2010-05-02 at 21:35 -0700, Alexander Dunlap wrote:
f ys = let xs = (1:ys) in last xs
uses the partial function "last". Rewriting it in the "non-partial style" gives
f ys = case (1:ys) of [] -> Nothing xs -> Just (last xs)
I guess it is more like f :: Num a => [a] -> a f = fromMaybe 1 . safeLast Regards

Ivan Miljenovic
On 3 May 2010 14:17, aditya siram
wrote: I'm a little confused about this too. I've seen many functions defined like: f x = (\s -> ...) which is a partial function because it returns a function and is the same as: f x s = ...
No, that's a partially applied function.
A partial function is one such as:
secondElement (_:x:_) = x
Note that it is only defined for lists with at least two elements, for any other list (i.e. singleton or empty) it will throw an error; -fwarn-incomplete-patterns (which is included in -Wall) tells you about these.
You can also argue that functions such as head are partial, as they explicitly throw an error if the input data isn't correct.
Partial functions are bad because if you accidentally use one the wrong way, your entire program crashes in a flaming wreck. It's much better to do something like this:
safeSecondElement (_:x:_) = Just x safeSecondElement _ = Nothing
This will work with all possible input types.
I don't think that safeSecondElement is worse than secondElement. I think it's better for the program to crash right away when you try to do something that doesn't make sense. Getting the secondElement of a list with one or less elements doesn't make sense, so you are definetely doing something wrong if you expected the list to have two elements, but it doesn't. It's best that the program crashes there, than propagate the error and crash somewhere else or, worse, not crash at all and give a wrong answer.

There are many ways to handle errors in Haskell.
The Maybe method is the simplest but it also comes with a little overhead,
since you always have to unpack the Maybe a value return, and finally I
often end up by doing something like :
maybe (error "Unexpected Nothing") id someSafeFunction
when you absolutely do not expect your code to return a Nothing here.
There's also the possibility to handle errors through a monad (e.g. using
Maybe as a monad), but all your functions must be designed as such.
I don't how the Error monad could be used there, I've never studied it...
I think using explicit exceptions is better in that case :
secondElement (_:x:_) = x
secondElement _ = error "secondElement: length < 2"
The major problem with exceptions is that Haskell has -- afaik -- no notion
of traceback (to know exactly where the exception was raised). For instance,
here, if secondElement is called more than once in your code, if you get the
exception, you won't be immediately able to tell which part of the code
raised it.
2010/5/3 Rafael Cunha de Almeida
On 3 May 2010 14:17, aditya siram
wrote: I'm a little confused about this too. I've seen many functions defined
Ivan Miljenovic
disse: like: f x = (\s -> ...) which is a partial function because it returns a function and is the same as: f x s = ...
No, that's a partially applied function.
A partial function is one such as:
secondElement (_:x:_) = x
Note that it is only defined for lists with at least two elements, for any other list (i.e. singleton or empty) it will throw an error; -fwarn-incomplete-patterns (which is included in -Wall) tells you about these.
You can also argue that functions such as head are partial, as they explicitly throw an error if the input data isn't correct.
Partial functions are bad because if you accidentally use one the wrong way, your entire program crashes in a flaming wreck. It's much better to do something like this:
safeSecondElement (_:x:_) = Just x safeSecondElement _ = Nothing
This will work with all possible input types.
I don't think that safeSecondElement is worse than secondElement. I think it's better for the program to crash right away when you try to do something that doesn't make sense.
Getting the secondElement of a list with one or less elements doesn't make sense, so you are definetely doing something wrong if you expected the list to have two elements, but it doesn't. It's best that the program crashes there, than propagate the error and crash somewhere else or, worse, not crash at all and give a wrong answer. _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Rafael Cunha de Almeida wrote:
I don't think that safeSecondElement is worse than secondElement. I think it's better for the program to crash right away when you try to do something that doesn't make sense.
Getting the secondElement of a list with one or less elements doesn't make sense, so you are definetely doing something wrong if you expected the list to have two elements, but it doesn't. It's best that the program crashes there, than propagate the error and crash somewhere else or, worse, not crash at all and give a wrong answer.
There are two different issues at stake here, one is reactive and the other is proactive. You're responding to the reactive question: how do I deal with unexpected errors when they arise? But those who argue against partial functions are usually responding to the proactive question: how do I prevent unexpected errors from arising in the first place? By allowing partial functions we allow for the introduction of unexpected errors, which then forces us to handle the reactive question of what to do when those errors show up. But if partial functions are avoided, then there are no unexpected errors (of that sort) which could ever show up. We already have to deal with bottom, that's a fact of life for general-purpose programming. But partial functions allow us to introduce bottom in new ways. When we have bottom meaning just non-termination, then we have a good idea about how we should treat it. But once we start introducing other bottoms for things like non-exhaustive pattern matching or recoverable exceptions, then it's no longer clear what exactly bottom means. This in turn makes it hard to know whether the semantics of the program match the semantics we want. In particular, I'm a fan of semantics which say my program will not crash. Sure, crashing is better than silently corrupting everything; but not crashing is better than crashing. -- Live well, ~wren

G'day all.
Quoting aditya siram
I'm a little confused about this too. I've seen many functions defined like: f x = (\s -> ...) which is a partial function because it returns a function and is the same as: f x s = ...
Off the top of my head the State monad makes extensive use if this style. Is this bad?
It's not inherently bad style. In the State/Reader monads, the passed-in parameter is part of the type: type State s a = s -> (s,a) f :: Foo -> State s a Since f has only one argument according to its type signature, it makes sense to give it only argument in its rules as well: f x = \s -> ... This re-enforces the fact that s is an implementation detail of State. It is an error in Haskell to have multiple rules for a function with different arities. So this, for example, is illegal: f True = length f False [] = 3 f False (_:_) = 2 If there were a good reason for writing the first rule in a point-free manner, it would be perfectly reasonable to rewrite this using an explicit lambda. There are also performance issues which may be relevant. GHC doesn't break up lambdas to do let-floating, so if that's what you want, you may need to express it manually: data TreeSet a = Null | Singleton a | Doubleton a a | Branch a TreeSet TreeSet elem :: (Ord a) => TreeSet a -> a -> Bool elem Null = \p -> False elem (Singleton p1) = \p -> p == p1 elem (Doubleton p1 p2) = \p -> p == p1 || p == p2 elem (Branch p' l r) = let elem_l = elem l elem_r = elem r in \p -> if p < p' then elem_l p else elem_r p Like all situations where you've got more than one way to express something, which one you pick depends on what you're trying to say. Cheers, Andrew Bromage

On 03/05/2010, at 06:02, Jaco van Iterson wrote:
I was just wondering what methods are best to design/model the software in bigger projects when you are planning to use Haskell. Is there no difference compared to other languages? Are there any Haskell tools?
In addition to what Don said, here are a couple of things I've learned. This is just from personal experience so YMMV. Design in Haskell is much more often bottom-up than in, say, traditional OO where it's frequently top-down all the way. I believe this is mainly due to purity. When you have some kind of global state, your design process often has to be top-down because of intricate interactions between program components which modify that state. Designing Haskell software tends to involve much fewer diagrams than OO. Your most important design tool is the type system. You can often express large chunks of your design through types and have the compiler check and enforce them. Fiddling with types is often part of the design process and should be treated accordingly. If you stumble on a useful design pattern, think about how to encode it in the type system (this is quite different from OO patterns). Higher-order functions and type classes are very powerful tools for reducing coupling and for implementing "design patterns". Prototyping is very cheap and easy. Writing prototypes and playing with them in ghci allows you to see how your subsystems will behave and adjust the design accordingly. In general, you ought to write code (esp. type signatures) while designing. Some libraries/subsystems will evolve into or start out as EDSLs. This is good and should be encouraged. Identifying EDSLs that would be useful for implementing your software is an important step in the design process. Roman
participants (17)
-
Adam Vogt
-
aditya siram
-
ajb@spamcop.net
-
Alexander Dunlap
-
Bradford Larsen
-
Conor McBride
-
Don Stewart
-
Edgar Z. Alvarenga
-
Gregory Collins
-
Ivan Miljenovic
-
Jaco van Iterson
-
Limestraël
-
Maciej Piechotka
-
Rafael Cunha de Almeida
-
Roman Leshchinskiy
-
Sebastian Fischer
-
wren ng thornton