Ramin wrote:

...no matter how many tutorials I read, I find the only kind of monad I can write is the monad that I copied and pasted from the tutorial... I am writing a mini-database... The query itself is stateful... The query may also make updates to the records. I also thought of trying to re-write my query algorithm to somehow use "Control.Monad.State.Strict" instead of my own query type, but then I wouldn't get to write my own monad!

Just Control.Monad.State should work fine. Daniel Fischer wrote:

But this looks very much like an application well suited for the State monad (or a StateT). So why not use that?

I agree with Daniel. If you want to learn about the deeper theory of the inner workings of monads, that's great - go ahead, and have fun! But to solve your problem in practice, you don't need that level of knowledge. All you need to know about is get, put, modify, and liftIO. The StateT monad is really simple to use. In general, practical software is higher quality when it uses existing standard libraries. There is no more reason to re-invent the StateT monad than there is to re-invent anything else in the libraries. Among the multitude of monad tutorials out there, I wonder how many of them draw a clear line between what you need to understand to design monads, and what you need to understand just to use them. There's a huge difference in complexity. Like most things, it is best to use monads for a while and get comfortable with them before trying to learn how to design them and build them. Regards, Yitz

On 9/3/08, wren ng thornton wrote:

If you want the datatype to be strict in state and rec, then you should add strictness annotations to those fields directly:

data Query state rec = Query !state !rec

The novice programmer scatters strictness annotations to and fro like dust in the wind. The average programmer understands that annotating a field's strictness injudiciously is like discarding the finger pointing at the moon when you might still need it to scratch yourself. The master programmer does not add strictness annotations, for she has not yet run the profiler. Cheers, TIm -- Tim Chevalier * http://cs.pdx.edu/~tjc * Often in error, never in doubt "Science fiction is not predictive; it is descriptive." --Ursula K. Le Guin

On 9/4/08, Jake Mcarthur wrote:

My guess would be that a master usually adds strictness annotations as documentation rather than as optimizations.

I'm no master, but I've never encountered a situation where strictness annotations would be useful as documentation, nor can I imagine one. That's because optimization *is* the only reason why programmers should care about strictness information. IMO, arguing that programmers should care at all amounts to conceding that default laziness is treacherous. Cheers, Tim -- Tim Chevalier * http://cs.pdx.edu/~tjc * Often in error, never in doubt "Stupidity combined with arrogance and a huge ego will get you a long way." -- Chris Lowe

On Sep 4, 2008, at 11:23 PM, Jake Mcarthur wrote:

To quote a blog article[1] I wrote in June,

And of course I would forget to link the article. My bad. [1] http://geekrant.wordpress.com/2008/06/23/misconceptions/ - Jake McArthur

On Sep 4, 2008, at 11:40 PM, Tim Chevalier wrote:

But why not write your types like: data Stream a = Cons a <Stream a> data Vector3 a = Vector3 a a a in a hypothetical call-by-value language where the <> annotation denotes a lazily evaluated data structure? Does it matter? If it does, then why? If it doesn't, then what would you conclude about whether a language should encourage laziness or strictness?

It doesn't matter, and I don't think it says anything about whether we should encourage lazy-by-default or strict-by-default. Two lazy algorithms tend to compose well and result in a lazy algorithm. A lazy algorithm can compose with a strict algorithm in two different ways. One way is for the lazy algorithm to control the strict algorithm, in which case the strict algorithm is either invoked or not invoked, resulting in a lazy algorithm. The other way is for the strict algorithm to control the lazy algorithm, in which case the strict algorithm requests the data it needs from the lazy algorithm as it needs it, resulting in a strict algorithm. Finally, two strict algorithms may also compose, which results in a strict algorithm. No matter how you slice it, none of the above scenarios are necessarily bad. Each of the four permutations of laziness and strictness for two composed algorithms are necessary for different situations. Laziness and strictness work in tandem with each other to construct whole programs. We Haskellers like laziness by default because we find that it encourages us to consider laziness to solve our problems more often than in call-by-need languages, not because it is somehow "superior" to strictness. That is the strongest argument I can think of to be made in favor of lazy-by-default.

both laziness and strictness exist in strict languages, as well. What if, as a thought experiment, you tried substituting "laziness" for "strictness" in that paragraph of your essay?

I think the same points would apply, honestly. Do you believe they would change in some way? - Jake McArthur

On Sep 5, 2008, at 12:45 AM, Tim Chevalier wrote:

On 9/4/08, Jake Mcarthur wrote:

...

Two lazy algorithms tend to compose well and result in a lazy algorithm. A lazy algorithm can compose with a strict algorithm in two different ways. One way is for the lazy algorithm to control the strict algorithm, in which case the strict algorithm is either invoked or not invoked, resulting in a lazy algorithm. The other way is for the strict algorithm to control the lazy algorithm, in which case the strict algorithm requests the data it needs from the lazy algorithm as it needs it, resulting in a strict algorithm. Finally, two strict algorithms may also compose, which results in a strict algorithm.

No matter how you slice it, none of the above scenarios are necessarily bad. Each of the four permutations of laziness and strictness for two composed algorithms are necessary for different situations. Laziness and strictness work in tandem with each other to construct whole programs.

You say lazy algorithms are good because they compose well. In Haskell, does an algorithm that operates on data structures that have strict components have that property?

The algorithm itself can still be lazy in that it might not evaluate the strict data structures at all, and thus can still have that property. However, I did not mean to necessarily emphasize the composability of lazy functions over the composability of strict functions. Both, theoretically, can compose equally well. Composition for laziness and strictness just mean slightly different things as they pertain to control flow and data flow.

...

We Haskellers like laziness by default because we find that it encourages us to consider laziness to solve our problems more often than in call-by-need languages, not because it is somehow "superior" to strictness. That is the strongest argument I can think of to be made in favor of lazy-by-default.

So you don't believe that laziness is superior to strictness, or versa; I don't, either. But you do say it's good to be encouraged to use laziness more often. Why? You mention compositionality above as a possible reason, in reply to which, see above.

Mostly because, at least in this part of history, it is easy in many languages to forget about lazy evaluation altogether. I don't think it's a matter of encouraging laziness _over_ strictness, as they are both beneficial, just a matter encouraging the programmer to _think_ in terms of laziness and strictness more often. - Jake McArthur

Jules Bean writes:

...

...

...

On Sep 4, 2008, at 10:19 AM, Tim Chevalier wrote:

...

...

...

...

The master programmer does not add strictness annotations, for she has not yet run the profiler.

...

The compiler will certainly be able to infer the strictness itself

As far as I am aware this statement is false.

The master programmer does not add strictness annotations, she improves the strictness analyser. -k -- If I haven't seen further, it is by standing in the footprints of giants

On 9/4/08, John Dorsey wrote:

I'm no master either, but I'd argue that if we promise new programmers that they don't need to care about strictness, we thereby ensure that default laziness is treacherous.

A year or two ago, ISTR that *most* of the newbie-generated traffic in the cafe was about atrocious performance of naive programs due to strict/lazy concerns. I think it was scaring people away.

I think it's debatable what the various causality relationships might be here.

Adding strictness can improve asymptotic space performance, as an example. Is there a reason to think this won't always be true? Honest question, since I don't know nearly enough about strictness analysis to guess how good it'll be some day.

Adding strictness can also worsen asymptotic space (and time) performance. That's one reason why we use a lazy language at all. Strictness analysis is an approximation to the problem of determining what parts of a program can be evaluated strictly without changing their meaning, because if we had a perfect solution to that problem, we could solve the halting problem. Cheers, Tim -- Tim Chevalier * http://cs.pdx.edu/~tjc * Often in error, never in doubt "There are no difficult problems, just unfortunate notations." -- Alfonso Gracia-Saz

On Thu, 2008-09-04 at 08:19 -0700, Tim Chevalier wrote:

On 9/3/08, wren ng thornton wrote:

...

If you want the datatype to be strict in state and rec, then you should add strictness annotations to those fields directly:

data Query state rec = Query !state !rec

The novice programmer scatters strictness annotations to and fro like dust in the wind. The average programmer understands that annotating a field's strictness injudiciously is like discarding the finger pointing at the moon when you might still need it to scratch yourself. The master programmer does not add strictness annotations, for she has not yet run the profiler.

This attitude is wrong. Many potentially significant performance problems can easily be spotted and solved during construction without affecting the readability of the code, problems that would be much harder to diagnose at the end running a profiler. This is especially crucial for library code. The users of the library may be the ones that find the easily resolved space leak your profiling didn't reveal and now they can't do anything about it until a new version is released e.g. Data.Map.insertWith. A performance problem that renders your code unusable is a bug and catching it early or not making it in the first place is much better than catching it late. A (highly related) analogy would be telling Scheme programmers (or Haskell programmers for that matter) not to use to tail recursive code until a profiler tells them to and transforming to a tail recursive style is much more intrusive than adding a strictness annotation. Highly competent Haskell programmers add strictness annotations relatively systematically. The details of mixing eager and lazy code is one of the significant contributions to the pragmatics of programming lazy functional languages have made. At another extreme, things like Chris Okasaki's data structures rely on a specific balance of eagerness and laziness. Also, it is easier (as in not impossible) to turn a strict in the elements data structure into a lazy one than the other way around. Eager by default or lazy by default are both have (actually dual) pitfalls that are best solved by a laziness or strictness annotation respectively. There is no need to walk into those pitfalls with your eyes wide open.

On Fri, 2008-09-05 at 20:11 -0700, Tim Chevalier wrote:

On Fri, Sep 5, 2008 at 7:23 PM, Derek Elkins wrote:

...

This attitude is wrong. Many potentially significant performance problems can easily be spotted and solved during construction without affecting the readability of the code, problems that would be much harder to diagnose at the end running a profiler. This is especially crucial for library code. The users of the library may be the ones that find the easily resolved space leak your profiling didn't reveal and now they can't do anything about it until a new version is released e.g. Data.Map.insertWith. A performance problem that renders your code unusable is a bug and catching it early or not making it in the first place is much better than catching it late.

Library writers don't write applications that use their code as part of the testing process?

They don't use their libraries in every conceivable way.

...

A (highly related) analogy would be telling Scheme programmers (or Haskell programmers for that matter) not to use to tail recursive code until a profiler tells them to and transforming to a tail recursive style is much more intrusive than adding a strictness annotation.

Tail recursion isn't always a win in Haskell.

I'm very well aware of that. Due to the extremely close relationship with when to tail recursion and when to use laziness, your original statement almost obligates you to also believe that "masters" don't write tail recursive code until after profiling.

I'm not much in touch with the Scheme world, so can't speak to that.

Stick in any strict functional language or any other strict language (preferably higher order, but that isn't really important) that also supports tail call optimization.

...

Highly competent Haskell programmers add strictness annotations relatively systematically. The details of mixing eager and lazy code is one of the significant contributions to the pragmatics of programming lazy functional languages have made. At another extreme, things like Chris Okasaki's data structures rely on a specific balance of eagerness and laziness.

Also, it is easier (as in not impossible) to turn a strict in the elements data structure into a lazy one than the other way around.

Eager by default or lazy by default are both have (actually dual) pitfalls that are best solved by a laziness or strictness annotation respectively. There is no need to walk into those pitfalls with your eyes wide open.

That may be, but are the holes that one falls into due to unexpected laziness and the ones that one falls into due to unexpected strictness equally numerous? Are they equally deep?

As I said, this particular issue is a duality. For at least a certain class of problems, the ones most relevant here, the holes are equally deep and equally numerous. What is correct in one language is a problem in the other. For examples and more discussion see: http://lambda-the-ultimate.org/node/2273#comment-40156 These last questions, though, are both flawed and irrelevant. They are flawed because the issue isn't which is the default: having both available -at all- makes you vulnerable to the pitfalls of both. Here's a good example: compose = foldr (.) id or should it be compose = foldl' (.) id maybe something else? The fact that Haskell has both strict and non-strict functions makes neither of these implementations correct. It is irrelevant because all that needs to be shown is that there exist -any- problem that can easily be identified and fixed with a strictness annotation. The Haskell implementation of factorial in the above has such a bug, it can be fixed with the addition of two characters, $!, I don't need a profiler to tell me this, and for every standard numeric type there will be no change in semantics (where "semantics" here does not include "performance" aspects.)