
Hi, I'd like to propose to include the following function in Data.List: join :: [a] -> [[a]] -> [a] join x xs = concat (intersperse x xs) I find that every time I use intersperse I also concat the result. And it seems that I'm not alone: http://www.google.com/codesearch?q=file%3A%5C.hs+intersperse&btnG=Search+Code A clear majority of all the uses of intersperse also use concat on the result. I think it seems worthwhile to give a name to that idiom. The name 'join' was taken from Data.ByteString, where a similar function exists. All the best, Josef Svenningsson

josef.svenningsson:
Hi,
I'd like to propose to include the following function in Data.List: join :: [a] -> [[a]] -> [a] join x xs = concat (intersperse x xs)
I find that every time I use intersperse I also concat the result. And it seems that I'm not alone: http://www.google.com/codesearch?q=file%3A%5C.hs+intersperse&btnG=Search+Code A clear majority of all the uses of intersperse also use concat on the result. I think it seems worthwhile to give a name to that idiom.
Oh, that's a nice trick for working out apis. Good idea!
The name 'join' was taken from Data.ByteString, where a similar function exists.
Seems reasonable. Want to submit a darcs patch, and maybe a quickcheck property or two? -- Don

On 10/21/06, Donald Bruce Stewart
josef.svenningsson:
The name 'join' was taken from Data.ByteString, where a similar function exists.
Seems reasonable. Want to submit a darcs patch, and maybe a quickcheck property or two?
But wait! There is also a join in Control.Monad!

On 10/21/06, Samuel Bronson
On 10/21/06, Donald Bruce Stewart
wrote: josef.svenningsson:
The name 'join' was taken from Data.ByteString, where a similar function exists.
Seems reasonable. Want to submit a darcs patch, and maybe a quickcheck property or two?
But wait! There is also a join in Control.Monad!
Good point. But I don't really see that as a problem. Don't you think the two 'join's can co-exist? It's just that I happen to like 'join' very much. But perhaps you have an alternative suggestion? Cheers, Josef

Hello Josef, Saturday, October 21, 2006, 5:43:24 PM, you wrote:
It's just that I happen to like 'join' very much. But perhaps you have an alternative suggestion?
this pair is named split/join in other languages too and quickcheck properties should be stolen from FPS, i think :) prop_joinsplit c xs = join ([c]) (split c xs) == id xs -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com

On Sat, Oct 21, 2006 at 03:43:24PM +0200, Josef Svenningsson wrote:
But wait! There is also a join in Control.Monad!
Good point. But I don't really see that as a problem. Don't you think the two 'join's can co-exist?
Data.List and Control.Monad have no conflicting names at the moment. On the other hand, you have a problem only when you use the conflicting name in your code. Perhaps it wouldn't be so bad, but I tend to use (concat . intersperse sep) a lot, and if I used join instead of that, I would have to hide "join" from all my Control.Monad imports. How about: separateWith sepBy (like in Parsec) joinWith or in the spirit of concatMap: concatIntersperse but that's a bit long. Best regards Tomasz

Neil Mitchell wrote:
At Linspire, we have been calling this function 'consperse'.
I've been calling this 'intercat' for some time now.
It would be nice if we could find a name that made it clear that: words = intercat " " I don't like wordsBy, but maybe there is some other suffix that would do the trick.

Hello Clifford, Saturday, October 21, 2006, 11:16:02 PM, you wrote:
It would be nice if we could find a name that made it clear that:
words = intercat " "
unwords? :) i use joinWith in my program -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com

On 10/21/06, Donald Bruce Stewart
josef.svenningsson:
I find that every time I use intersperse I also concat the result. And it seems that I'm not alone: http://www.google.com/codesearch?q=file%3A%5C.hs+intersperse&btnG=Search+Code A clear majority of all the uses of intersperse also use concat on the result. I think it seems worthwhile to give a name to that idiom.
Oh, that's a nice trick for working out apis. Good idea!
Thanks!
The name 'join' was taken from Data.ByteString, where a similar function exists.
Seems reasonable. Want to submit a darcs patch, and maybe a quickcheck property or two?
Well, I do have commit rights myself so I can apply whatever we agree on. Or did you mean that it would be nice with some public scrutiny of the patch? Where would the quickcheck properties go, btw? Cheers, Josef

Let's get join and split done! I want to see this function done this time. So let's try hard not to restart the big threads we've had on this topic in the last couple of years. So I'll just point to the archives. http://thread.gmane.org/gmane.comp.lang.haskell.glasgow.user/10136/focus=136... Getting bogged down in fiddly details will just derail this effort. See here. Perfectionists will not be tolerated! ;) http://haskell.org/haskellwiki/Protect_the_community Now, in summary, Josef: * Send a patch to libraries@, for a minimal implementation of the join function * And a patch for QuickCheck properites, to live under testsuite/tests/*. - e.g. for split . join == id words == join " " Now, the name, join conflicts, so: * consperse * connect * intercat * the problem with joinWith (or whatever) is that traditionally it would be joinBy So, I propose we start with just, say, 'connect', with some QC properties, and get that into base, before we get dragged out into a big discussion about the entire api design. -- Don

On 10/21/06, Donald Bruce Stewart
Let's get join and split done! [snip] Now, the name, join conflicts, so: * consperse * connect * intercat * the problem with joinWith (or whatever) is that traditionally it would be joinBy
If we're going to have a split or splitBy then wouldn't the most logical name be unsplit or unsplitBy? I think consperse and intercat are cute, but not that good because they don't really tell me what's going on. I think 'connect' is too vague and overloaded. So yeah, my vote is with unsplitBy. On the other hand, whoever implments it should get to choose the name and just be done with it :) Anyway, just some fuel for the fire :) Jason

Donald Bruce Stewart wrote:
* the problem with joinWith (or whatever) is that traditionally it would be joinBy
You mean as in zipBy, exitBy, parZipBy, insertBy, unionBy, differenceBy, fromListBy, showTreeBy, showListBy and applyBy? I see, that's certainly gonna be a problem. I think, joinWith is actually a good choice. The xxxBy functions take ordering functions as argument, and that doesn't apply to joinWith. Udo. -- Si Hoc Legere Scis Nimium Eruditionis Habes.

On 2006-10-22 at 12:50+1000 dons@cse.unsw.edu.au (Donald Bruce Stewart) wrote:
Now, the name, join conflicts, so: * consperse * connect * intercat * the problem with joinWith (or whatever) is that traditionally it would be joinBy
So, I propose we start with just, say, 'connect',
intercalate, surely? OED says of intercalate: transf. a. To insert or interpose something additional, extraneous, or out of the ordinary course, between the ordinary members of any series [...] Since there's already a word that means almost exactly what we want, why not use it? Jón -- Jón Fairbairn Jon.Fairbairn at cl.cam.ac.uk

Hello Jon, Sunday, October 22, 2006, 4:18:27 PM, you wrote:
intercalate, surely?
Since there's already a word that means almost exactly what we want, why not use it?
it's a rare word that is unknown for english-guests like me -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com

On 2006-10-22 at 18:11+0400 Bulat Ziganshin wrote:
Hello Jon,
Sunday, October 22, 2006, 4:18:27 PM, you wrote:
intercalate, surely?
Since there's already a word that means almost exactly what we want, why not use it?
it's a rare word that is unknown for english-guests like me
В противоположность "Monad" например? (Apologies for what is probably appalling grammar!) More seriously, the more I think about it, the more I wonder if giving names to things that have such a short expression as “((concat .) . intersperse)” might not be counterproductive. Whatever we call it, even if it's a word in an English dictionary, it's meaning in Haskell becomes an additional learning burden on a programmer who whishes to understand programmes that use it. “(concat.).intersperse” might cause some head scratching at first, but everything involved is something that the programmer will have to learn anyway, and if its use is frequent, it will become idiomatic. Jón -- Jón Fairbairn Jon.Fairbairn at cl.cam.ac.uk

Hello Jon, Sunday, October 22, 2006, 7:16:28 PM, you wrote:
intercalate, surely? it's a rare word that is unknown for english-guests like me
В противоположность "Monad" например?
(Apologies for what is probably appalling grammar!)
it's correct :) and you found a great example - this word from math language is sense-less for peoples whose native language is not Math and makes all sorts of confusion. naming it Combinator or Strategy or smth like this will cut learning curve
More seriously, the more I think about it, the more I wonder if giving names to things that have such a short expression as “((concat .) . intersperse)” might not be counterproductive.
Whatever we call it, even if it's a word in an English dictionary, it's meaning in Haskell becomes an additional learning burden on a programmer who whishes to understand programmes that use it. “(concat.).intersperse” might cause some head scratching at first, but everything involved is something that the programmer will have to learn anyway, and if its use is frequent, it will become idiomatic.
once you've learned smth, it looks really easy and you may wonder why other stupid peoples can't understand this immediately. but recall how much time of Haskell learning was required for you to make itself familiar with such idioms? from viewpoint of readability for as much people as possible i prefer join or joinWith. join used in ruby and, i think, in perl. joinWith is at least readable. all other ways, imho, leads to building "secret language" which foreigners will need to learn without getting any advantages -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com

On 10/22/06, Bulat Ziganshin
from viewpoint of readability for as much people as possible i prefer join or joinWith. join used in ruby and, i think, in perl. joinWith is at least readable. all other ways, imho, leads to building "secret language" which foreigners will need to learn without getting any advantages
joinWith sounds nice to me ;-).

Summary: further thought has convinced me that the proper thing to do is not to name this in a library (or other very short functions). On 2006-10-22 at 21:09+0400 Bulat Ziganshin wrote:
Sunday, October 22, 2006, 7:16:28 PM, you wrote:
intercalate, surely? it's a rare word that is unknown for english-guests like me
But if you type "define:intercalate" into google (даже google.ru), you find what it means.
В противоположность "Monad" например? (Apologies for what is probably appalling grammar!) it's correct :)
More by good luck than good management, I fear¹...
and you found a great example - this word from math language is sense-less for peoples whose native language is not Math and makes all sorts of confusion. naming it Combinator or Strategy or smth like this will cut learning curve
Perhaps, but at the end of that new learning curve, you wouldn't know what a Monad is, which would leave you the poorer.
More seriously, the more I think about it, the more I wonder if giving names to things that have such a short expression[...] might not be counterproductive.
[...] “(concat.).intersperse” might cause some head scratching at first, but everything involved is something that the programmer will have to learn anyway, and if its use is frequent, it will become idiomatic.
once you've learned smth, it looks really easy and you may wonder why other stupid peoples can't understand this immediately. but recall how much time of Haskell learning was required for you to make itself familiar with such idioms?
That's rather orthogonal to my point. Suppose we chose "join"; learners would find that join is like it is in other languages, and then find that there's also a join in Monad, and that would be confusing, which is counterproductive. If we call it intercalate, there's the slight benefit of widening some people's English vocabulary, but that's not worth that much. If someone knows what they want to do (and already knows the word), then they might well check to see if there's a function called intercalate in the libraries, which again is a slight advantage. If we don't give it a name, learning to read “concat . intersperse something” (which is more common than the formulation I gave above) means learning intersperse, concat and compose -- all of which are invaluable Haskell. But if we call it whateverify, there will still be programmes around that use concat with intersperse (I don't plan to go through all my old code and use the new name, and I doubt many would). Furthermore, for a seasoned Haskell programmer who wants to (for example) concatenate a list l with commas between the elements, what springs to mind is “concat (intersperse ", ") l” and it comes to mind so quickly that looking for a library function to do it would seem like an onerus additional task. Of course, if that construction occurred several times in a programme, they might choose to call it "commas" or something for conciseness, but that sort of naming for brevity is a separate issue from putting something in a library. To consider a different example, while you might reasonably expect a library function to compute the RMS of a list of values, I don't think you would bother to look for a function to compute the sum of the squares of a list, because it's just “sum . map (**2)”, and that's easy enough to read.
from viewpoint of readability for as much people as possible i prefer join or joinWith. join used in ruby and, i think, in perl. joinWith is at least readable. all other ways, imho, leads to building "secret language" which foreigners will need to learn without getting any advantages
but that makes my point for me. Giving names in libraries to short functions that have a pretty much unique natural way of being written amounts to a "secret language" (it wouldn't matter if it was just intercalate or whatever, one new function more or less doesn't make much difference, but that's a small pile of stones argument). When I said that if it's used often enough it becomes idiomatic, I meant that one learns to read it without thinking -- that applies to intercalate or joinWith or whatever, but the difference is if you /don't/ already know what it means, joinWith has to be looked up (or guessed, again I'm addressing the general case of naming short things rather than joinWith which might be guessed correctly quite often), whereas “concat . intersperse foo” can be worked out by knowing relatively primitive things. For me, the main thing that comes out of the postings offering different alternative names is that the "correct" name isn't obvious. Way back before Haskell when I was a beginner functional programmer designing Ponder, I had my own "obvious" name for it, and that name hasn't been suggested -- moreover, I had a function called "join_with" that did something else! So someone who is looking for a function to do this won't know what name to look for, and will quite likely write it in less time than it takes to Hoogle for it by type. That means that you'll have to learn what “concat . intersperse foo” means anyway, in addition to learning joinWith. Certainly complicated enough constructions deserve library entries, and I admit I don't know where the break even point is, but I'm pretty sure it comes above two words and some punctuation -- unless it's something fundamental like composition. [1] Russian grammar is remarkably difficult for this native English speaker. Conversely, I'm well aware that certain constructions in English (eg "would have been going to go") aren't easy for Russian speakers. -- Jón Fairbairn Jon.Fairbairn at cl.cam.ac.uk

Hello Jon, Monday, October 23, 2006, 7:41:56 PM, you wrote:
intercalate, surely? it's a rare word that is unknown for english-guests like me
But if you type "define:intercalate" into google (даже google.ru), you find what it means.
thank you. how about using russian words, you can find it too? ;) -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com

I remember that there was one version of Logo that was translated to
Bulgarian. You can write things like:
напред 60 надясно 60 надясно 60 надясно 60
to draw a square with the turtle. :-)
Cheers,
Krasimir
On 10/23/06, Bulat Ziganshin
Hello Jon,
Monday, October 23, 2006, 7:41:56 PM, you wrote:
intercalate, surely? it's a rare word that is unknown for english-guests like me
But if you type "define:intercalate" into google (даже google.ru), you find what it means.
thank you. how about using russian words, you can find it too? ;)
-- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com
_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

Jon Fairbairn
If we don't give it a name, learning to read “concat . intersperse something” (which is more common than the formulation I gave above) means learning intersperse, concat and compose -- all of which are invaluable Haskell. But if we call it whateverify, there will still be programmes around that use concat with intersperse [...]
I agree. I've always been slightly annoyed at the presence of 'concatMap' since its replacement is so trivial. Another reason for a special function could be efficiency. You could possibly have the case that intersperse (or map) construct new strings that only get copied and discarded by the subsequent concat. (A sufficiently smart compiler will probably be able to optimize this away, though) -k -- If I haven't seen further, it is by standing in the footprints of giants

Jon,
I'm sorry for my late reply.
I do sympathize with your general argument about introducing new small
functions in the library. However, I'd would like to once again argue
for the particular function intercalate. I don't think I was as clear
as I could have been in my original post.
My argument for intercalate builds on the fact that intersperse seems
to be the wrong abstraction. Recall that I supplied some evidence that
intersperse is only used together with concat:
http://www.google.com/codesearch?q=file%3A%5C.hs+intersperse&btnG=Search+Code
Except for pretty printing, intersperse always comes together with
concat. For me this suggests that intersperse is the wrong
abstraction. The right one is intercalate. Therefore, in the best of
worlds, intercalate should replace intersperse (or intersperse should
be moved to the pretty printing library, the seemingly only place it's
used). Of course that's not going to happen, at least not in the API.
But perhaps in the minds of Haskell programmers. We will hopefully
learn to use intercalate instead, and intersperse will slowly drift
out of our mental toolbox of library functions.
On 10/23/06, Jon Fairbairn
If we don't give it a name, learning to read "concat . intersperse something" (which is more common than the formulation I gave above) means learning intersperse, concat and compose -- all of which are invaluable Haskell. But if we call it whateverify, there will still be programmes around that use concat with intersperse (I don't plan to go through all my old code and use the new name, and I doubt many would). Furthermore, for a seasoned Haskell programmer who wants to (for example) concatenate a list l with commas between the elements, what springs to mind is "concat (intersperse ", ") l" and it comes to mind so quickly that looking for a library function to do it would seem like an onerus additional task. Of course, if that construction occurred several times in a programme, they might choose to call it "commas" or something for conciseness, but that sort of naming for brevity is a separate issue from putting something in a library.
This would be a valid argument, if it were for the fact that intersperse really was invaluable, which it seems not to be.
Certainly complicated enough constructions deserve library entries, and I admit I don't know where the break even point is, but I'm pretty sure it comes above two words and some punctuation -- unless it's something fundamental like composition.
This argument is orthogonal to why I want intercalate. I hope that intercalate will replace intersperse, at least in my head, because it seems to be the right abstraction. Apart from that I agree with what you say. I know that many people will argue that intersperse is somehow more primitive or more natural than intercalate. As for being more primitive, that's not true. Both functions are interdefinable. But I agree that intersperse does feel more natural and simpler in a sense. Yet, if it always requires some padding around it to make it work (i.e. concat) then I would argue that it is *too* simple. Better use the function which doesn't need the padding. All the best, Josef Svenningsson

On Fri, Oct 27, 2006 at 05:56:55PM +0200, Josef Svenningsson wrote:
My argument for intercalate builds on the fact that intersperse seems to be the wrong abstraction. Recall that I supplied some evidence that intersperse is only used together with concat: http://www.google.com/codesearch?q=file%3A%5C.hs+intersperse&btnG=Search+Code
Except for pretty printing, intersperse always comes together with concat.
It's also useful with foldr (.) id when showing things.

On 2006-10-27 at 17:56+0200 "Josef Svenningsson" wrote:
My argument for intercalate builds on the fact that intersperse seems to be the wrong abstraction. Recall that I supplied some evidence that intersperse is only used together with concat: http://www.google.com/codesearch?q=file%3A%5C.hs+intersperse&btnG=Search+Code
Mostly, not only.
For me this suggests that intersperse is the wrong abstraction. The right one is intercalate.
I don't agree with the methodology... Firstly, while extant code tells you how things have been used, it doesn't tell you how they /should/ be used. Second, I suspect that the majority of current Haskell code was written by novices -- I don't mean anyone any disrespect, it's just that it takes quite a long time to become an expert, the number of new novices has been increasing rapidly and the people who have known Haskell long enough to be experts probably nolonger write so much code: they get their students to do it :-). [How many of the cases you found should have been foldr (.) id etc?]
I know that many people will argue that intersperse is somehow more primitive or more natural than intercalate.
I don't know about natural. Looking back at the 25 year old Ponder list_ops library, I find that I defined join_with filler l1 [] = l1 join_with filler l1 l2 = l1 ++ filler ++ l2 and then flatten_with filler = foldr (join_with filler) [] which is another way of writing intercalate.
As for being more primitive, that's not true. Both functions are interdefinable.
Right. So your motive is more complicated than it appeared to me. I missed that first time round, sorry. The problem we have is that intersperse x == intercalate [x] . map (:[]) and intercalate x = concat . intersperse x While it seems easier to me to work out the second than the first, I don't know if that is simply due to habituation. I can add that the second /looks/ simpler. So I /suspect/ (unprovably) that were intercalate the preferred form, people would write intersperse in several different ways, in contrast to the present state of affairs where “concat . intersperse splod” seems already to have become idiomatic. What would convince me of your case would be someone saying that intercalate is the <oojit> of the <weeble> category while intersperse enjoys no such property... ... or maybe I was right back then, and the natural primitive is something like join_with -- an operation to use with foldr? -- Jón Fairbairn Jon.Fairbairn at cl.cam.ac.uk

On 10/29/06, Jon Fairbairn
On 2006-10-27 at 17:56+0200 "Josef Svenningsson" wrote:
My argument for intercalate builds on the fact that intersperse seems to be the wrong abstraction. Recall that I supplied some evidence that intersperse is only used together with concat: http://www.google.com/codesearch?q=file%3A%5C.hs+intersperse&btnG=Search+Code
Mostly, not only.
OK, it's mostly. If you had quoted a bit more of what I wrote I did actually nuance my statement a bit, acknowledge pretty printing code as an exception.
For me this suggests that intersperse is the wrong abstraction. The right one is intercalate.
I don't agree with the methodology... Firstly, while extant code tells you how things have been used, it doesn't tell you how they /should/ be used. Second, I suspect that the majority of current Haskell code was written by novices -- I don't mean anyone any disrespect, it's just that it takes quite a long time to become an expert, the number of new novices has been increasing rapidly and the people who have known Haskell long enough to be experts probably nolonger write so much code: they get their students to do it :-). [How many of the cases you found should have been foldr (.) id etc?]
Of course there is a risk of being influenced by bad coding principle when using this methodology. But do you think that this is what happened in this particular case? I challenge you to go through all the search results and report any bad code that you find so that we can avoid being influenced by that. I did that myself before sending my proposal but given what you write above perhaps you should double check. Then there is all the good code that *hasn't* been written but perhaps should have been and therefore was excluded from the search. If you feel that that's what happened here I ask you to put forward concrete examples which we can learn from. As for you second objection. To me it seems that because you don't like the outcome of the code search you declare the programmers who wrote the code to be immature.
I know that many people will argue that intersperse is somehow more primitive or more natural than intercalate.
I don't know about natural. Looking back at the 25 year old Ponder list_ops library, I find that I defined
join_with filler l1 [] = l1 join_with filler l1 l2 = l1 ++ filler ++ l2
and then
flatten_with filler = foldr (join_with filler) []
which is another way of writing intercalate.
Nice!
As for being more primitive, that's not true. Both functions are interdefinable.
Right. So your motive is more complicated than it appeared to me. I missed that first time round, sorry.
The problem we have is that
intersperse x == intercalate [x] . map (:[])
and
intercalate x = concat . intersperse x
While it seems easier to me to work out the second than the first, I don't know if that is simply due to habituation. I can add that the second /looks/ simpler. So I /suspect/ (unprovably) that were intercalate the preferred form, people would write intersperse in several different ways, in contrast to the present state of affairs where "concat . intersperse splod" seems already to have become idiomatic.
I agree that the second looks simpler. So you're suspecting that people would define their own intersperse in many different ways. I don't see how that fits in our current discussion. No one is proposing to remove intersperse. If people do find a need for intersperse then they should go ahead and use it. My argument is that people *don't* use intersperse, at least not without concat, and that that idiom will hopefully be replaced with intersperse.
What would convince me of your case would be someone saying that intercalate is the <oojit> of the <weeble> category while intersperse enjoys no such property...
Yes. That would indeed be a nice thing. I know that the reasoning I use to argue for intercalate is one which we are not used to in the Haskell community. We want our stuff to be firmly based in mathematics and not some accidental choice of idiom by some unknown body of programmers. But I personally find intercalate very useful and common enough to warrant a name. And apparently so does many others.
... or maybe I was right back then, and the natural primitive is something like join_with -- an operation to use with foldr?
Perhaps? But what does it mean to be right in this situation? I'd like to come to some sort of conclusion. Judging from the response I got from my suggestion to include intercalate in the library it seems there is a clear majority in favour. And then there is you and a few others who disagree. With the new submission guidelines we should strive for a consensus about patches. How do you suggest we proceed? And I didn't make this any easier by unilaterally applying the patch before we had a consensus. But that patch is not the final word. There is still time to make things right. All the best, Josef Svenningsson

On 2006-10-30 at 14:03+0100 "Josef Svenningsson" wrote:
On 10/29/06, Jon Fairbairn
wrote: On 2006-10-27 at 17:56+0200 "Josef Svenningsson" wrote:
My argument for intercalate builds on the fact that intersperse seems to be the wrong abstraction. Recall that I supplied some evidence that intersperse is only used together with concat: http://www.google.com/codesearch?q=file%3A%5C.hs+intersperse&btnG=Search+Code
Mostly, not only.
OK, it's mostly.
I probably shouldn't have made that point, it was a knee-jerk reaction to seeing a false statement.
If you had quoted a bit more of what I wrote I did actually nuance my statement a bit, acknowledge pretty printing code as an exception.
There's also this: hugs98-Feb2001/lib/hugs/GenericPrint.hs - 12 identical 313: intersperse :: a -> [a] -> [a] intersperse x (y:ys@(_:_)) = y : x : intersperse x ys intersperse x ys = ys for__ :: Monad m => [a] -> (a -> m ()) -> m () -> m () for__ xs f inc = sequence $ intersperse inc $ map f xs which while it's in some printing code isn't an instance of the kind of exception that you are making.
For me this suggests that intersperse is the wrong abstraction. The right one is intercalate.
I don't agree with the methodology... Firstly, while extant code tells you how things have been used, it doesn't tell you how they /should/ be used. Second, I suspect that the majority of current Haskell code was written by novices -- I don't mean anyone any disrespect, it's just that it takes quite a long time to become an expert, the number of new novices has been increasing rapidly and the people who have known Haskell long enough to be experts probably nolonger write so much code: they get their students to do it :-). [How many of the cases you found should have been foldr (.) id etc?]
Of course there is a risk of being influenced by bad coding principle when using this methodology. But do you think that this is what happened in this particular case?
No, I'm not talking about bad code in this case -- I don't see anything at all wrong with writing “concat . intersperse "blah"”, I'm complaining about the methodology in general. In this case I could even argue that your data supports my position! How many of those occurrences of “concat . intersperse x” are in the body of a definition of intercalate (by whatever name)? Out of 100 results, there's the join in ByteString, joinList in Pugs-6.0.1 (but then there are instances of the idiom rather than joinList in later versions of Pugs), a join_with in FreeArc, and that's it. So most of those programmers (and my guess would be that it's not really enough to mean much) were sufficiently happy with intersperse not to bother abstracting away from it. There /are/ several definitions of the form “spacify = concat . instersperse " "”, which also seems reasonable.
I challenge you to go through all the search results and report any bad code that you find so that we can avoid being influenced by that.
Given that that's not my complaing, I don't think I need to, but since I had to go through to see how many were defining intercalate I did find this comment "this is hideously broken, actually" in one, which might just suggest bad code ;-)
Then there is all the good code that *hasn't* been written but perhaps should have been and therefore was excluded from the search. If you feel that that's what happened here I ask you to put forward concrete examples which we can learn from.
That just seems like an unfair challenge. How am I supposed to come up with a programme that one might want to write for which intersperse is exactly the right abstraction? I've no idea what programme that might be!
As for you second objection. To me it seems that because you don't like the outcome of the code search you declare the programmers who wrote the code to be immature.
Now that's a gross mischaracterisation of my second point! Let me recast it in a way that insults noone but me: Every line of code that I have ever written was written by someone less experienced that I am now. So I don't want to use that code in deciding how I /should/ write code in future. This is essentially the is/ought problem in philosophy. If your argument had been that all these uses could have been written more concisely using intercalate, I couldn't have disputed it, but I would still have made the argument I did about not giving names to small functions. Instead, you are trying to argue that they ought to have been written using intercalate because it's the right abstraction, and whether or not that conclusion is true, the data neither supports nor undermines it.
I don't know about natural. Looking back at the 25 year old Ponder list_ops library, I find that I defined
join_with filler l1 [] = l1 join_with filler l1 l2 = l1 ++ filler ++ l2
and then
flatten_with filler = foldr (join_with filler) []
which is another way of writing intercalate.
Nice!
But I should add that it's a subtly different function, in case anyone didn't notice.
As for being more primitive, that's not true. Both functions are interdefinable.
Right. So your motive is more complicated than it appeared to me. I missed that first time round, sorry.
The problem we have is that
intersperse x == intercalate [x] . map (:[])
and
intercalate x = concat . intersperse x
[...] I /suspect/ (unprovably) that were intercalate the preferred form, people would write intersperse in several different ways, in contrast to the present state of affairs where "concat . intersperse splod" seems already to have become idiomatic.
I agree that the second looks simpler.
So you're suspecting that people would define their own intersperse in many different ways. I don't see how that fits in our current discussion. No one is proposing to remove intersperse.
Oh, I thought what you were proposing was that intersperse would fall out of use, so there would then be no point in having it in a library. Having both of them is definitely an instance of more vocabulary for little gain.
If people do find a need for intersperse then they should go ahead and use it. My argument is that people *don't* use intersperse, at least not without concat, and that that idiom will hopefully be replaced with intersperse.
But why? You hope for this, yet I thought you agreed with my general point that adding library names for short idiomatic constructions was counterproductive.
What would convince me of your case would be someone saying that intercalate is the <oojit> of the <weeble> category while intersperse enjoys no such property...
Yes. That would indeed be a nice thing. I know that the reasoning I use to argue for intercalate is one which we are not used to in the Haskell community. We want our stuff to be firmly based in mathematics and not some accidental choice of idiom by some unknown body of programmers.
Just so. But there are lots of languages where the stuff is based on accidental choices of idioms by unknown bodies, and I "[h]ates the lot of [th]em", so I don't want Haskell to go that way. I'm really uneasy at the idea that we should be in favour of rapid changes to libraries; I'd much rather they were developed after a good deal of argument about the mathematical properties.
But I personally find intercalate very useful and common enough to warrant a name. And apparently so does many others.
[...]
I'd like to come to some sort of conclusion. Judging from the response I got from my suggestion to include intercalate in the library it seems there is a clear majority in favour. And then there is you and a few others who disagree. With the new submission guidelines we should strive for a consensus about patches. How do you suggest we proceed?
For the case of intercalate, if everyone who is in favour of its inclusion has read my arguments and remains unswayed, then it should surely go in. I'd like to think that the points I'm making would have some effect, but if they don't, at least I get my choice of name! Jón -- Jón Fairbairn Jon.Fairbairn at cl.cam.ac.uk

On 10/30/06, Jon Fairbairn
Just so. But there are lots of languages where the stuff is based on accidental choices of idioms by unknown bodies, and I "[h]ates the lot of [th]em", so I don't want Haskell to go that way. I'm really uneasy at the idea that we should be in favour of rapid changes to libraries; I'd much rather they were developed after a good deal of argument about the mathematical properties.
So, just because nobody has figured out what the "<weeble>" category is, you don't want it? That seems like a silly reason... next you will be asking what mathematical construct "Show" relates to!

"Samuel Bronson"
On 10/30/06, Jon Fairbairn
wrote: Just so. But there are lots of languages where the stuff is based on accidental choices of idioms by unknown bodies, and I "[h]ates the lot of [th]em", so I don't want Haskell to go that way. I'm really uneasy at the idea that we should be in favour of rapid changes to libraries; I'd much rather they were developed after a good deal of argument about the mathematical properties.
So, just because nobody has figured out what the "<weeble>" category is, you don't want it?
You need to read more carefully. I said "what would convince me ..." I didn't say that the absence of a proof convinces me otherwise. Everything /else/ I've been arguing convinces me otherwise.
That seems like a silly reason... next you will be asking what mathematical construct "Show" relates to!
Of course not. But I do care that Show has showsPrec and that elements of the type ShowS has useful compositional properties. These things were thought about carefully. -- Jón Fairbairn Jon.Fairbairn@cl.cam.ac.uk

On 10/30/06, Jon Fairbairn
On 2006-10-30 at 14:03+0100 "Josef Svenningsson" wrote:
On 10/29/06, Jon Fairbairn
wrote: I don't agree with the methodology... Firstly, while extant code tells you how things have been used, it doesn't tell you how they /should/ be used. Second, I suspect that the majority of current Haskell code was written by novices -- I don't mean anyone any disrespect, it's just that it takes quite a long time to become an expert, the number of new novices has been increasing rapidly and the people who have known Haskell long enough to be experts probably nolonger write so much code: they get their students to do it :-). [How many of the cases you found should have been foldr (.) id etc?]
Of course there is a risk of being influenced by bad coding principle when using this methodology. But do you think that this is what happened in this particular case?
No, I'm not talking about bad code in this case -- I don't see anything at all wrong with writing "concat . intersperse "blah"", I'm complaining about the methodology in general. In this case I could even argue that your data supports my position! How many of those occurrences of "concat . intersperse x" are in the body of a definition of intercalate (by whatever name)? Out of 100 results, there's the join in ByteString, joinList in Pugs-6.0.1 (but then there are instances of the idiom rather than joinList in later versions of Pugs), a join_with in FreeArc, and that's it. So most of those programmers (and my guess would be that it's not really enough to mean much) were sufficiently happy with intersperse not to bother abstracting away from it.
There /are/ several definitions of the form "spacify = concat . instersperse " "", which also seems reasonable.
I challenge you to go through all the search results and report any bad code that you find so that we can avoid being influenced by that.
Given that that's not my complaing, I don't think I need to, but since I had to go through to see how many were defining intercalate I did find this comment "this is hideously broken, actually" in one, which might just suggest bad code ;-)
Then there is all the good code that *hasn't* been written but perhaps should have been and therefore was excluded from the search. If you feel that that's what happened here I ask you to put forward concrete examples which we can learn from.
That just seems like an unfair challenge. How am I supposed to come up with a programme that one might want to write for which intersperse is exactly the right abstraction? I've no idea what programme that might be!
You said that you didn't like the methodology. What I wanted to know was exactly how you thought that it had gone wrong in this particular case. The challenges I gave you was only meant to apply if you subscribed to the particular objections that I suggested. I'm sorry if I was unclear about that.
As for you second objection. To me it seems that because you don't like the outcome of the code search you declare the programmers who wrote the code to be immature.
Now that's a gross mischaracterisation of my second point! Let me recast it in a way that insults noone but me: Every line of code that I have ever written was written by someone less experienced that I am now. So I don't want to use that code in deciding how I /should/ write code in future. This is essentially the is/ought problem in philosophy. If your argument had been that all these uses could have been written more concisely using intercalate, I couldn't have disputed it, but I would still have made the argument I did about not giving names to small functions. Instead, you are trying to argue that they ought to have been written using intercalate because it's the right abstraction, and whether or not that conclusion is true, the data neither supports nor undermines it.
How do you find new abstractions? A simplified description might be: You look at previous work and see if there is a pattern emerging. When you spot a pattern you give that a name. And there you have it, a new abstraction. Do you disagree with this way of working? We learn from what we have done before and find new and better ways of doing the same thing. We spot new structures and abstractions and learn how to think about things at a higher level. I think it is a very sound principle to look at old code and find new a better ways of writing it. Indeed, if I wrote the code I was probably a lesser programmer at the time I wrote it. But that doesn't mean that I cannot learn from the code and improve it, does it? Or do I misunderstand you again?
I don't know about natural. Looking back at the 25 year old Ponder list_ops library, I find that I defined
join_with filler l1 [] = l1 join_with filler l1 l2 = l1 ++ filler ++ l2
and then
flatten_with filler = foldr (join_with filler) []
which is another way of writing intercalate.
Nice!
But I should add that it's a subtly different function, in case anyone didn't notice.
Indeed. To me flatten_with seems less usable than intercalate. But that's just my initial impression. Do you have thoughts on its merits vs. intercalate?
So you're suspecting that people would define their own intersperse in many different ways. I don't see how that fits in our current discussion. No one is proposing to remove intersperse.
Oh, I thought what you were proposing was that intersperse would fall out of use, so there would then be no point in having it in a library. Having both of them is definitely an instance of more vocabulary for little gain.
If people do find a need for intersperse then they should go ahead and use it. My argument is that people *don't* use intersperse, at least not without concat, and that that idiom will hopefully be replaced with intersperse.
But why? You hope for this, yet I thought you agreed with my general point that adding library names for short idiomatic constructions was counterproductive.
I agree that my line of reasoning can be a bit confusing here. The gist of the problem is that, due to backwards compatibility, intersperse will not be removed from the library in the foreseeable future. So we're going to have both intersperse and intercalate in the library. Unfortunate, indeed. But in the minds of the programmer (at least if his name is Josef) intersperse will be replaced with intercalate, and intersperse will fall out of use. I agree that it is not optimal to have both intersperse and intercalate in the library. But I'd rather have that than not have intercalate since I find it so useful.
But I personally find intercalate very useful and common enough to warrant a name. And apparently so does many others.
[...]
I'd like to come to some sort of conclusion. Judging from the response I got from my suggestion to include intercalate in the library it seems there is a clear majority in favour. And then there is you and a few others who disagree. With the new submission guidelines we should strive for a consensus about patches. How do you suggest we proceed?
For the case of intercalate, if everyone who is in favour of its inclusion has read my arguments and remains unswayed, then it should surely go in. I'd like to think that the points I'm making would have some effect, but if they don't, at least I get my choice of name!
OK. I do appreciate your comments and they have been duly noted. And yes, you did get to choose the name :-) All the best, Josef

On 2006-11-02 at 13:01+0100 "Josef Svenningsson" wrote:
On 10/30/06, Jon Fairbairn
wrote: On 2006-10-30 at 14:03+0100 "Josef Svenningsson" wrote: is essentially the is/ought problem in philosophy. If your argument had been that all these uses could have been written more concisely using intercalate, I couldn't have disputed it, but I would still have made the argument I did about not giving names to small functions. Instead, you are trying to argue that they ought to have been written using intercalate because it's the right abstraction, and whether or not that conclusion is true, the data neither supports nor undermines it.
How do you find new abstractions? A simplified description might be: You look at previous work and see if there is a pattern emerging. When you spot a pattern you give that a name. And there you have it, a new abstraction.
But not all patterns deserve names. To give a reductio-ad-absurdam sort of analogy, suppose someone looked at a lot of C and came to the conclusion that “for (i=0, i++, ...)” occurred in 90% of programmes. Were they to reason that it would be worth #defining a macro FORI(...) and using that instead, I suspect that the suggestion would be roundly dismissed.
Do you disagree with this way of working? We learn from what we have done before and find new and better ways of doing the same thing. We spot new structures and abstractions and learn how to think about things at a higher level. I think it is a very sound principle to look at old code and find new a better ways of writing it. Indeed, if I wrote the code I was probably a lesser programmer at the time I wrote it. But that doesn't mean that I cannot learn from the code and improve it, does it?
Of course not, but you have to be careful what you deduce from the old code, particularly if you are reasoning from "most code is like this", because that doesn't tell you whether most code should be like that or most code should have been written differently. Another silly analogy: if most pancakes are served with maple syrup, that doesn't mean that we should always add maple syrup to the mix. It certainly suggests it as a worthwhile experiment, but since I eat my pancakes with fish, I could hardly be expected to approve. That exact argument doesn't apply to the specific case of intercalate, because here it's possible to get the maple syrup back out of the pancake.
Or do I misunderstand you again?
I've been trying to conduct several distinct arguments, some general and some specific here, and I think it's confusing enough that I'd better list them. The first is the general one that adding a name for a small function is only worthwhile if it really is a new abstraction, otherwise it increases vocabulary without increasing expressiveness. The second is the general one that one has to be very careful about what one deduces from "most code is like this", which applies here in the specific form that, OK, you've found that most occurrences of intersperse are with concat, but what can you deduce from that? Certainly, all those programmes could be very slightly shorter had we put intercalate in the standard prelude in the first place. But it doesn't tell us whether we should have done that. Then there's the specific question of whether intercalate is actually a better abstraction than intersperse. “concat . intersperse ", "” is hardly difficult to read, and I find “intercalate [", "] . map (:[])” (for the reverse case) slightly harder to read -- though I don't know whether that's down to habituation, so we're just arguing intuitions here. Finally, there's another specific one, that whatever the merits of intercalate, intersperse got there first, which means that unless there's a really strong argument that intercalate is the better abstraction, it's better to leave things alone rather than increase the number of ways of expressing the same concept.
I agree that it is not optimal to have both intersperse and intercalate in the library. But I'd rather have that than not have intercalate since I find it so useful.
People always have the option of defining it for themselves. I certainly think that in the end, if this function does get defined lots of times (which, supposing it's not put in library, it might now that we've discussed it so much) then making sure that they all have the same name is a Good Thing. But if it's put in a library and a significant proportion go on writing “concat . intersperse "..."”, it won't be. And once something has gone into a library, it's awfully hard (read impossible) to get back out again. Jón -- Jón Fairbairn Jon.Fairbairn at cl.cam.ac.uk

Hi
But if it's put in a library and a significant proportion go on writing "concat . intersperse "..."", it won't be.
That just means that not enough people are using Dr Haskell on their code :) http://www-users.cs.york.ac.uk/~ndm/projects/drhaskell.php Thanks Neil

Jon Fairbairn
Then there's the specific question of whether intercalate is actually a better abstraction than intersperse. "concat . intersperse" is hardly difficult to read,
I have found over the years, that pretty-much every time I write "intersperse" in a program, I get a type error. It almost always turns out that I really wanted "concat . intersperse", and so I write that instead. So perhaps this just demonstrates that I have the wrong idea about what intersperse is supposed to do. But the very fact that I keep making the same mistake suggests that there is indeed a natural and intuitive function hiding there, and I keep reaching for it, but with the wrong name. Regards, Malcolm

On Thu, 9 Nov 2006 17:27:39 +0000 you wrote:
Jon Fairbairn
wrote: Then there's the specific question of whether intercalate is actually a better abstraction than intersperse. "concat . intersperse" is hardly difficult to read,
I have found over the years, that pretty-much every time I write "intersperse" in a program, I get a type error. It almost always turns out that I really wanted "concat . intersperse", and so I write that instead.
That is certainly an interesting datapoint. Unfortunately I don't believe I've ever made that particular mistake, so we cancel each other out ;-). But it raises a point about collecting data for this kind of argument. What we want is a history of the mistakes people made; /then/ we could argue from searches in such histories which constructs were more error-prone. Until you wrote this I was thinking that perhaps a search of version-control histories would do, but since people don't (I hope!) check in code that doesn't compile, your sort of problem goes unrecorded.
So perhaps this just demonstrates that I have the wrong idea about what intersperse is supposed to do. But the very fact that I keep making the same mistake suggests that there is indeed a natural and intuitive function hiding there, and I keep reaching for it, but with the wrong name.
I suspect what's natural and intuitive differs from person to person too much for us to make progress on that front without the sort of data that I say above is hard to get, which is why I'd prefer to see arguments about mathematical properties. Largely pointless rumination follows. One of the properties is the type; I can immediately think of three types that the "intuitive" function might have: intersperse :: a -> [a] -> [a] inter_concat :: [a] -> [a] -> [a] -- not the best name? intercalate :: [a] -> [[a]] -> [a] from that I should prefer intersperse on the grounds that it has a simpler type, but when I tried writing each in terms of the other, I found that the definitions of intersperse and intercalate in terms of inter_concat were both nice and obvious. Defining inter_concat in terms of intersperse is a bit messy, and inter_concat in terms of intercalate only slightly less so, which bend me towards the belief that inter_concat is the most primitive! The only conclusion I can really draw from this is that the argument that wins for me is still: intersperse got there first, it may not be ideal, but the grounds for adding another are too weak. Jón -- Jón Fairbairn Jon.Fairbairn@cl.cam.ac.uk 31 Chalmers Road jf@cl.cam.ac.uk Cambridge CB1 3SZ +44 1223 570179 (after 14:00 only, please!)

Jón Fairbairn wrote:
Largely pointless rumination follows.
One of the properties is the type; I can immediately think of three types that the "intuitive" function might have:
intersperse :: a -> [a] -> [a] inter_concat :: [a] -> [a] -> [a] -- not the best name? intercalate :: [a] -> [[a]] -> [a]
from that I should prefer intersperse on the grounds that it has a simpler type, but when I tried writing each in terms of the other, I found that the definitions of intersperse and intercalate in terms of inter_concat were both nice and obvious. Defining inter_concat in terms of intersperse is a bit messy, and inter_concat in terms of intercalate only slightly less so, which bend me towards the belief that inter_concat is the most primitive!
With inter_concat, I have to guess what it might mean; the other two are clear to me. One major use of intercalate is of course for show functions; since I still tend to define my show instances via showsPrec rather than show, I found the following function useful --- the type is rather general, but the (unreflected) name gives away its normal use: sepShowsList :: (s -> s) -> (a -> s -> s) -> [a] -> s -> s sepShowsList sep shows [] = id sepShowsList sep shows [x] = shows x sepShowsList sep shows (x:xs) = shows x . sep . sepShowsList sep shows xs The obvious generalisation is to monoids (ignoring the fact that sepShowsList also has a map built in), and we see that the underlying algebraic effect is to use |mSep| to produce a semigroup from a monoid element: type SemiGroup s = s -> s -> s -- must be associative mSep :: Monoid m => m -> SemiGroup m mSep x y z = y `mappend` x `mappend` z mSepConcat :: Monoid m => m -> [m] -> m mSepConcat = foldr1 . mSep Note that the last is a point-free definition ;-)
The only conclusion I can really draw from this is that the argument that wins for me is still: intersperse got there first, it may not be ideal, but the grounds for adding another are too weak.
The one I have been defining again and again is intercalate, mostly as: sepConcat :: [a] -> [[a]] -> [a] If it is not provided by the library, I will define it again, and many others will probably do the same, only perhaps with different names ... Wolfram

On Friday 10 November 2006 12:34, kahl@cas.mcmaster.ca wrote:
Jón Fairbairn wrote:
[snip]
The only conclusion I can really draw from this is that the argument that wins for me is still: intersperse got there first, it may not be ideal, but the grounds for adding another are too weak.
The Haskell standard libraries are not, and have never been, a minimal basis lacking redundancy. Are you also opposed to the existence of mapM f = sequence . map f and other such useful goodies? I use mapM because it eliminates parens and reduces line length for a programming pattern I use a lot. This increases redability of my code: win. Codifying this pattern by placing it in the standard libraries means that most people use the same name for the same concept, increasing overall readability: win. I'm don't thing your criteria (which seems to be related to the ease with which the function is defined in terms of other, pre-existing functions) is a good discriminator for what should and what should not be in the standard libraries.
The one I have been defining again and again is intercalate, mostly as:
sepConcat :: [a] -> [[a]] -> [a]
If it is not provided by the library, I will define it again, and many others will probably do the same, only perhaps with different names ...
In my view, this is the only point that really matters in this debate. Programmers find this concept useful and they name it. This has been done independently by multiple people persuing diverse ends. The fact that this function isn't in the standard lib means that programmers name it different things and, in the long run, this harms readability across Haskell code. We should codify this idea. This function should be in the standard libs. I personally don't much care what it's called.
Wolfram
-- Rob Dockins Talk softly and drive a Sherman tank. Laugh hard, it's a long way to the bank. -- TMBG

On 11/10/06, Robert Dockins
I'm don't thing your criteria (which seems to be related to the ease with which the function is defined in terms of other, pre-existing functions) is a good discriminator for what should and what should not be in the standard libraries.
I don't think so either. It certainly doesn't seem to categorize most of the libraries. Is this another case of the bikeshed problem?

pe, 2006-11-10 kello 12:51 -0500, Robert Dockins kirjoitti:
The Haskell standard libraries are not, and have never been, a minimal basis lacking redundancy. Are you also opposed to the existence of ... Programmers find this concept useful and they name it. This has been done ... We should codify this idea. This function should be in the standard libs. I personally don't much care what it's called.
And as was said, what's natural and intuitive differs from person to person meaning IMO that it's better to support different ways of doing and thinking things. Thus, both in the libs sounds fine. br, Isto

I simply don't have the stamina to follow up to all the objections to my messages. I'm posting this here in the thread because it's a convenient point, not because Robert's message troubles me particularly. Evidently I'm not getting my point across, so I'll give one more try and then call it a day (for ten years or so, then I can look again and see if the wind has changed). 1. I don't want to remove any extant library functions -- these have already been subsumed into the collective consciousness. 2. I'm not opposed to intercalate in particular, it's just an instance of what I perceive to be a growing problem. 3. I'm not going to be strongly opposed to any particular suggested small function -- each usually has the merit of reducing the number of tokens in the code, which with all else being equal would be a good thing. All else is not equal, though. I was going to post something more in the thread on adding on, in that ultimately “<comparison> `on` fst” is more readable than “equating”, “comparing” and friends, simply because of the quantity of names. But even of those I cannot get really excited in my denouncement. I don't know for certain about anyone else, but I for one have a limited capacity for learning arbitrary names. I'm pretty sure there's a limit for most people, but for some the limit is so large that learning all the names in all the Haskell libraries that there will be in the future won't be a problem. Such folk aren't going to be inconvenienced in the long run, but for feeble minded people such as myself, the application of a brake to the proliferation of names would be an important gain. To draw a parallel, for most of my life I've been intermittently trying to learn Chinese characters. There are so many, and although there's a degree of compositionality from radicals to whole characters, a great deal of the relationship between any complex character and its meaning is arbitrary. Consequently I've yet to learn enough that I can read a single sentence in Chinese. Six years ago, I had a Russian lodger and thought it would be fun to learn a bit of Russian. There's only a few more characters in Russian than English, so I learnt them all in a couple of days (not the order, though: that's arbitrary). After that, I found that Russian words are often composed of prefixes that modify the meaning of smaller words¹. Because of this, even without trying at all hard I can already read quite a bit of Russian. The difference is that with Chinese characters there's a whole lot of arbitrary symbols to learn before you can get anywhere, while with Russian there seems to be fewer arbitrary symbols at each level. (This isn't just me, by the way: Chinese characters are a significant obstacle to literacy in China -- it's even possible that was originally deliberate -- so they use pinyin) What I'm advocating is that when deciding whether to put something into a library we make sure that it's worth the extra effort to the reader, so that reading Haskell doesn't become as hard as reading Chinese -- only as hard as Russian ;-). To twist Gauss's snippy remark to Wilson into a rule, what I want is no new notations without new notions. If you can name something without controversy so that the ordinary English (or mathematical) reading both tells the reader immediately what the function does and is obvious to the writer looking for it (the latter is the lesser), then it can go in a library. Otherwise we have to consider: is there a more powerful function that does the job? is it possible to define the same function at a higher type? is the name going to be more useful at a different meaning? is the named version /really/ more readable than the expression it stands for? And it's not always going to be possible to answer those questions without considerable work. Particularly the last case: it's quite hard, especially for a beginning (or intermediate) programmer, to escape the idea that by naming a short bit of code one has made it simpler. Part of the trouble is that, having had to think about the thing long enough to want to name it, the name becomes (for that programmer) a shorhand for the entire process of discovering the combination. Ten years later, the discovery doesn't seem so radical, reading the short bit of code is straightforward and remembering what the name means has become difficult. On 2006-11-10 at 12:51EST Robert Dockins wrote:
The Haskell standard libraries are not, and have never been, a minimal basis lacking redundancy.
I'm not sure how you got the idea that that's what I want. There are several reasons that can make naming something short worthwhile. Take “sum”. It's code is short, but there are decisions to be made (foldl or foldr &c) that mean that the name abstracts something useful away from the code. The things I've been objecting to are pretty much the only way of writing something in terms of predefined functions.
Are you also opposed to the existence of
mapM f = sequence . map f
That's an interesting case. I'm not, because of point (1) above, but I reserve the right to whine about it. Having a naming convention is better than having completely arbitrary names, but the fact that a convention was needed should raise suspicion. Given that we had concatMap already, sequenceMap would have been a more easily interpreted name -- and then some people would wonder, why bother with the name? It took me a while to realise that mapM isn't some form of [f]map, and longer still to notice that fmap for Monads is inexplicably called liftM.
and other such useful goodies? I use mapM because it eliminates parens
You save one pair.
and reduces line length
by a small constant
for a programming pattern I use a lot. This increases redability of my code: win.
Yes, but there's a reduction in readability too, though it's harder to notice once you've learned the name mapM, and so long as the number of names like that in libraries is small the loss of readability in this way is negligible. But if we go on adding such names, it will become a real problem.
Codifying this pattern by placing it in the standard libraries means that most people use the same name for the same concept, increasing overall readability: win.
On the other hand, if there were no names for it and people wrote the same thing in each case, it would be just as readable. (Do I need to repeat that point (1) makes this specific case moot?). We need names for medium to large concepts, not for really small ones.
Programmers find this concept useful and they name it. This has been done independently by multiple people persuing diverse ends. The fact that this function isn't in the standard lib means that programmers name it different things and, in the long run, this harms readability across Haskell code.
Well, for the specific case of intercalate, the data Joseph presented doesn't support that -- there were lots of instances of the code written out, but only a few where this was the body of a definition, and some of the definitions were for a specific separator. Finding a concept useful and naming it is something that programmers do, but it's far from clear to me that it's always what they should do. I can't present a real life case, but it seems to me that there will be times when it obscures what's going on. For example suppose “weeble . sort” occurs often and is given the name “foo” and that “reverse . whiffle” has similarly been named “bar”. Now, a programmer who writes “foo . bar” might be quite happy that this does the right thing and fail to notice that “weeble . sort . reverse . whiffle” simplifies to “weeble . sort . whiffle” (given appropriate conditions). This is particularly likely when a library function has been found without looking at the code. Nor do I think that programmers naming things differently is /necessarily/ a loss, either. Sometimes the name says something about the intentional meaning that use of a name from a library would not. Finally, we have to question how far one has to search to find the definition of a name. There are (a) names that one knows already (but there is a limit to how many of these there can be), (b) names defined in the module one is currently reading, and (c) unfamiliar names defined elsewhere that one has to look up. That is, I think, a list in order of difficulty, and the more names there are in libraries, the more times what was hoped to be an (a) becomes a (c). Jón [1] eg опасность means danger, безопасность is without+danger = safety, небезопасность is not-without-danger = insecurity. For some reason I find this amusing. -- Jón Fairbairn Jon.Fairbairn at cl.cam.ac.uk

I empathise with what you're saying, John, but I think it will fall upon deaf ears unless the community agrees that your points support one of the goals of the Haskell libraries project. I wrote some notes on formal consensus for free software projects a while ago ( http://bobstopper.livejournal.com/22939.html ) and I think one of my points is applicable here: without codified community principles, it's difficult to identify what is and isn't relevant discussion for the project. Clearly you consider the discussion relevent while many others don't. I think your points would carry much more weight if you didn't have to argue your case every time you raised them. For this, having a set of principles which the community adheres to would be a boon: it could clearly articulate whether the community as a whole is ultimately concerned with the proliferation of names for small functions thus ensuring that such proposals in future are judged in a consistent way without the need for superfluous discussion. I'd suggest this community write up a simple set of principles with measures to allow future proposals to ammend the principles. Start with the smaller, more obvious principles and later move to add principles that deal with the issues such as Jon is raising. On Wed, 2006-11-15 at 18:52 +0000, Jon Fairbairn wrote:
I simply don't have the stamina to follow up to all the objections to my messages. I'm posting this here in the thread because it's a convenient point, not because Robert's message troubles me particularly.
--
Robert Marlow

By the way, the proper way I should have suggested this is by writing a proposal with some suggested principles, but I don't feel I'm enough of an aged haskell library hacker to start making calls like that. If someone more experienced thinks this is a good idea it would be good if they could have a crack at writing up some principles or suggest some to me to write up. On Thu, 2006-11-16 at 09:20 +0900, Robert Marlow wrote:
I empathise with what you're saying, John, but I think it will fall upon deaf ears unless the community agrees that your points support one of the goals of the Haskell libraries project.
I wrote some notes on formal consensus for free software projects a while ago ( http://bobstopper.livejournal.com/22939.html ) and I think one of my points is applicable here: without codified community principles, it's difficult to identify what is and isn't relevant discussion for the project. Clearly you consider the discussion relevent while many others don't.
I think your points would carry much more weight if you didn't have to argue your case every time you raised them. For this, having a set of principles which the community adheres to would be a boon: it could clearly articulate whether the community as a whole is ultimately concerned with the proliferation of names for small functions thus ensuring that such proposals in future are judged in a consistent way without the need for superfluous discussion.
I'd suggest this community write up a simple set of principles with measures to allow future proposals to ammend the principles. Start with the smaller, more obvious principles and later move to add principles that deal with the issues such as Jon is raising.
On Wed, 2006-11-15 at 18:52 +0000, Jon Fairbairn wrote:
I simply don't have the stamina to follow up to all the objections to my messages. I'm posting this here in the thread because it's a convenient point, not because Robert's message troubles me particularly.
--
Robert Marlow

Jon Fairbairn wrote:
But not all patterns deserve names. To give a reductio-ad-absurdam sort of analogy, suppose someone looked at a lot of C and came to the conclusion that ???for (i=0, i++, ...)??? occurred in 90% of programmes. Were they to reason that it would be worth #defining a macro FORI(...) and using that instead, I suspect that the suggestion would be roundly dismissed.
Well, I presume you actually meant 'for( i=0; ... ; i++ )', an error which would have been avoided by using a macro. Also, a macro would have liberated you of the choice between '++i' and 'i++', where '++i' is often more efficient (at least in C++). And in C++, you're encouraged to use std::for_each anyway, which is the C++ way to spell 'mapM_'. Which would make your point kinda moot, but I'm not really convinced that arguments from a souped up assembly language can be used to make a point about Haskell.
Of course not, but you have to be careful what you deduce from the old code, particularly if you are reasoning from "most code is like this", because that doesn't tell you whether most code should be like that or most code should have been written differently.
Exactly! To state it explicitly: it seems, 'intersperse' is either used at type 'String' or in conjunction with pretty printing. Or in other words, either with pretty printing or with ugly printing. Therefore, the right thing to do is not to provide an ugly printing idiom, but to improve Text.PrettyPrint to the point where it is easier to use than plain strings. A tutorial should be enough to achieve that. In fact, if I had a better understanding of the behaviour of Text.PrettyPrint, and if there was a way to easily format tables, I'd never mess with strings for human readable output. Udo. -- It is better to have loved a short man than never to have loved a tall.

On 11/9/06, Udo Stenzel
Jon Fairbairn wrote:
But not all patterns deserve names. To give a reductio-ad-absurdam sort of analogy, suppose someone looked at a lot of C and came to the conclusion that ???for (i=0, i++, ...)??? occurred in 90% of programmes. Were they to reason that it would be worth #defining a macro FORI(...) and using that instead, I suspect that the suggestion would be roundly dismissed.
Well, I presume you actually meant 'for( i=0; ... ; i++ )', an error which would have been avoided by using a macro. Also, a macro would have liberated you of the choice between '++i' and 'i++', where '++i' is often more efficient (at least in C++). And in C++, you're encouraged to use std::for_each anyway, which is the C++ way to spell 'mapM_'. Which would make your point kinda moot, but I'm not really convinced that arguments from a souped up assembly language can be used to make a point about Haskell.
To further belabor the point...In the linux kernel they almost exactly do what Jon was pointing out as a dismissable solution. In the linux kernel they do it for the kernel's data structures. You can find macros galore for manipulating lists and iterating over them[1]. [1] http://isis.poly.edu/kulesh/stuff/src/klist/ Jason

On 2006-11-10 at 01:13PST "Jason Dagit" wrote:
To further belabor the point...In the linux kernel they almost exactly do what Jon was pointing out as a dismissable solution.
See my response to Udo for how I cocked up my "obviously dismissable example". But I can't resist responding to this bit:
In the linux kernel they do it for the kernel's data structures. You can find macros galore for manipulating lists and iterating over them[1].
I hope we're all agreed that using macros as a means of making code safer is like trying to mend the foundations of a skyscraper with string and sealing-wax. Jón -- Jón Fairbairn Jon.Fairbairn at cl.cam.ac.uk

On 2006-11-10 at 00:32+0100 Udo Stenzel wrote:
Jon Fairbairn wrote:
But not all patterns deserve names. To give a reductio-ad-absurdam sort of analogy, suppose someone looked at a lot of C and came to the conclusion that “for (i=0, i++, ...)” occurred in 90% of programmes. Were they to reason that it would be worth #defining a macro FORI(...) and using that instead, I suspect that the suggestion would be roundly dismissed.
Well, I presume you actually meant 'for( i=0; ... ; i++ )', an error which would have been avoided by using a macro.
If I don't know the order of the clauses in C loops, a macro isn't going to help much: I shouldn't programme in C full stop. Fortunately I don't, so that's OK. Unless you are seriously arguing that embedding the name of the loop control identifier in a macro is a good idea, it seems I pared down my "obviously silly" example to the point where the sillyness was no-longer obvious. I was trying to be too concise. I meant to say that "If someone found that 90% of control variables are called “i”, that wouldn't make it a good idea to have a macro “FORi” that expanded into “for (i,...)”" -- the incrementing by one was incidental.
Also, a macro would have liberated you of the choice between '++i' and 'i++', where '++i' is often more efficient (at least in C++). And in C++, you're encouraged to use std::for_each anyway, which is the C++ way to spell 'mapM_'. Which would make your point kinda moot, but I'm not really convinced that arguments from a souped up assembly language can be used to make a point about Haskell.
Evidently not. C is so clumsy that I messed up my trivial and silly example completely.
Of course not, but you have to be careful what you deduce from the old code, particularly if you are reasoning from "most code is like this", because that doesn't tell you whether most code should be like that or most code should have been written differently.
Exactly! To state it explicitly: it seems, 'intersperse' is either used at type 'String' or in conjunction with pretty printing. Or in other words, either with pretty printing or with ugly printing. Therefore, the right thing to do is not to provide an ugly printing idiom, but to improve Text.PrettyPrint to the point where it is easier to use than plain strings.
Right. If there is a problem, I don't think it's that “intercalate foo” is currently spelled “concat . intersperse foo”, and if there is something deeper we can do to mend the problem, I'm in favour of it. Actually, I'd start by deprecating Show, because for machine readability it's not efficient and for human readability it's too simplistic. Jón -- Jón Fairbairn Jon.Fairbairn at cl.cam.ac.uk

Hi
Actually, I'd start by deprecating Show, because for machine readability it's not efficient and for human readability it's too simplistic.
When I show values there are 3 categories they fall into: 1) Output in a way for debugging - the current deriving Show gives me this, but its a pain I have to add deriving Show everywhere just to get debugging - I'd rename it Raw, and have everything derive Raw automatically and without comment. 2) Ouputting simple things. If I have a list of things, a pretty printing combinator is overkill - and whats more I don't care if the lines wrap around. I'd much rather have a nice simple "make this a string". The current Show solves this perfectly. 3) Pretty printing. I have an Output class for this (since i like to use the deriving Show for debugging). Show is good at what it does, for use case number 2. I use interact (intercalate as it is now) and intersperse lots, in both variations. One is good, both are better! (Have they been committed now? I thought I saw that go through...) Thanks Neil

On 11/10/06, Neil Mitchell
I use interact (intercalate as it is now) and intersperse lots, in both variations. One is good, both are better!
(Have they been committed now? I thought I saw that go through...)
Yes, I have committed intercalate now. Josef

On Fri, 27 Oct 2006, Josef Svenningsson wrote:
My argument for intercalate builds on the fact that intersperse seems to be the wrong abstraction. Recall that I supplied some evidence that intersperse is only used together with concat: http://www.google.com/codesearch?q=file%3A%5C.hs+intersperse&btnG=Search+Code
Except for pretty printing, intersperse always comes together with concat.
I can tell you at least two non-concat applications of 'intersperse'. 1. Transforming a power series for 'f' to the onw for '\x -> f(x^2)' is simply 'intersperse 0'. 2. Upsampling a signal as needed for wavelet transform. However, you can argue that these applications are special cases of inserting (n-1) zeros between list items in order to multiply the length by a factor of n, and thus 'intersperse' doesn't scale well for this purpose.

On 2006-10-22, Jon Fairbairn
On 2006-10-22 at 12:50+1000 dons@cse.unsw.edu.au (Donald Bruce Stewart) wrote:
Now, the name, join conflicts, so: * consperse * connect * intercat * the problem with joinWith (or whatever) is that traditionally it would be joinBy
So, I propose we start with just, say, 'connect',
intercalate, surely?
A bit late for me to chime in, but I do like this. -- Aaron Denney -><-

On 24/10/06, Aaron Denney
On 2006-10-22, Jon Fairbairn
wrote: On 2006-10-22 at 12:50+1000 dons@cse.unsw.edu.au (Donald Bruce Stewart) wrote:
Now, the name, join conflicts, so: * consperse * connect * intercat * the problem with joinWith (or whatever) is that traditionally it would be joinBy
So, I propose we start with just, say, 'connect',
intercalate, surely?
A bit late for me to chime in, but I do like this.
Me too

On 10/22/06, Donald Bruce Stewart
Let's get join and split done!
I want to see this function done this time. So let's try hard not to restart the big threads we've had on this topic in the last couple of years. So I'll just point to the archives. http://thread.gmane.org/gmane.comp.lang.haskell.glasgow.user/10136/focus=136...
Getting bogged down in fiddly details will just derail this effort. See here. Perfectionists will not be tolerated! ;) http://haskell.org/haskellwiki/Protect_the_community
Well spoken.
Now, in summary, Josef: * Send a patch to libraries@, for a minimal implementation of the join function * And a patch for QuickCheck properites, to live under testsuite/tests/*. - e.g. for split . join == id words == join " "
Patches attached. I've also included a couple of properties that don't hold but which one might think hold. I'm not sure how useful it is to have that in the testsuite but I wanted to document them somewhere anyway.
Now, the name, join conflicts, so: * consperse * connect * intercat * the problem with joinWith (or whatever) is that traditionally it would be joinBy
So, I propose we start with just, say, 'connect', with some QC properties, and get that into base, before we get dragged out into a big discussion about the entire api design.
I think Jón's suggestion 'intercalate' was pretty sweet. I've chosen to use that name in my patch. Josef

On 10/22/06, Clifford Beshers
Josef Svenningsson wrote:
I think Jón's suggestion 'intercalate' was pretty sweet. I've chosen to use that name in my patch.
Could you add the definition (or a reference) to the patch? It would be nice to make sure people know that the Haskell community didn't make it up.
Do you think that's really necessary? We don't have explanations for intersperse, transpose, partition, insert, intersect, union etc. Nor do we make excuses for names like nub or mapAccumL. Is there a particular reason for explaining the name 'intercalate'? All the best, Josef

Josef Svenningsson wrote:
Do you think that's really necessary? We don't have explanations for intersperse, transpose, partition, insert, intersect, union etc. Nor do we make excuses for names like nub or mapAccumL. Is there a particular reason for explaining the name 'intercalate'?
Merely to point out to the new user that it is not a made up word. All the others are common terms. Perhaps it might be better simply to put it somewhere in the wiki.

Josef Svenningsson wrote:
On 10/22/06, Clifford Beshers
wrote: Josef Svenningsson wrote:
I think Jón's suggestion 'intercalate' was pretty sweet. I've chosen to use that name in my patch.
Could you add the definition (or a reference) to the patch? It would be nice to make sure people know that the Haskell community didn't make it up.
Do you think that's really necessary? We don't have explanations for intersperse, transpose, partition, insert, intersect, union etc. Nor do we make excuses for names like nub or mapAccumL.
The Haskell report used to explain 'nub' (my 1.1 report has the comment, but it seems to be lost in '98). Cheers, Simon

josef.svenningsson:
On 10/22/06, Donald Bruce Stewart
wrote: Let's get join and split done!
Patches attached.
I think J?n's suggestion 'intercalate' was pretty sweet. I've chosen to use that name in my patch.
Great. I think I actually prefer joinWith now. It's a lot easier for new programmers, and we chose that for ByteString too :) Can you perhaps change it to joinWith, and then commit, assuming no strong complaints emerge in the next day or two? Thanks , Don

On Mon, Oct 23, 2006 at 12:33:35PM +1000, Donald Bruce Stewart wrote:
josef.svenningsson:
On 10/22/06, Donald Bruce Stewart
wrote: Let's get join and split done!
Patches attached.
I think J?n's suggestion 'intercalate' was pretty sweet. I've chosen to use that name in my patch.
Great. I think I actually prefer joinWith now. It's a lot easier for new programmers, and we chose that for ByteString too :)
Can you perhaps change it to joinWith, and then commit, assuming no strong complaints emerge in the next day or two?
I like intercalate better too. 'join' is way too oveloaded as is. I think of lattices first (meet and join) and monads second.... intercalate means exactly what we want. I would also much prefer the more general intercalate :: Monoid w => a -> [a] -> w a intercalate x xs = mconcat (intersperse x xs) Although, now that we lost the Monoid instance for functions (which is very dismaying), it is less useful, as the monoid functions were very useful to build things up efficiently with (String -> String) as a type. John -- John Meacham - ⑆repetae.net⑆john⑈

Hi all, Bit of a sideways leap from joinWith versus intercalate (which is what you must do if you need to take time out after you matriculate but before you graduate), but I was inspired/provoked/reminded by this remark of John's... John Meacham wrote:
Although, now that we lost the Monoid instance for functions (which is very dismaying), it is less useful, as the monoid functions were very useful to build things up efficiently with (String -> String) as a type.
...(with which I rather disagree) to finish a note about programming with structure-indicating newtypes, like Endo. I thought I'd also remind you of a little of what we bought by wrapping the endofunction monoid, then making the function instance do pointwise lifting instead. I append it, for what it's worth. All the best Conor -----------------------------------------------------------------------
module Wise where
Perhaps it's quite foolish, but here's an exercise in modifying one's manners. I would not presume to use the title of this module as an adjective, but I rather like it as a /suffix/.
import Applicative import Traversable import Monoid
Firstly, we shall need a useful little type class to capture the pattern of using a special isomorph of a type to indicate not merely its raw data, but also its relevant structure.
class Unpack p u | p -> u where unpack :: p -> u (~~) :: (u -> p) -> p -> u (~~) _ = unpack
The idea is that the packed type |p| is a special-purpose relabelling of some duller unpacked type |u|. We thus expect |p| to determine |u|, but a given |u| may have many such |p|'s, eg |Sum Int| and |Product Int|, from |Data.Monoid|, wrapping Int to indicate an intended |Monoid| structure. Let's have an example:
newtype Id x = Id {getId :: x}
instance Unpack (Id x) x where unpack (Id x) = x
The |(~~)| method is provided as a uniform way to make an explicit inverse for the constructor. We can replace |getId| by |(Id ~~)|, and we never have to remember its name again, or even waste namespace on it in the first place. It's sometimes ambiguous and often unreadable to use unpack directly, when a specific instance is intended. Here, the structure I want to make explicit via |Id| is the notion of pure computation, normally left implicit. But pure computation iss as good a notion of computation as any other and quite often better.
instance Applicative Id where pure = Id Id f <*> Id s = Id (f s)
We use |Id| to get ordinary |fmap| from the more general |traverse|, explaining the manner of the traversal.
myFMap :: Traversable c => (s -> t) -> c s -> c t myFMap f cs = Id ~~ traverse (Id . f) cs
Recall that |Applicative (Const m)| if |Monoid m|. Let us have
instance Unpack (Const m x) m where unpack (Const c) = c
myCrush :: (Traversable c, Monoid m) => (s -> m) -> c s -> m myCrush m cs = Const ~~ traverse (Const . m) cs
Spotting the pattern? One more.
instance Unpack (Endo x) (x -> x) where unpack (Endo f) = f
myFold :: Traversable c => (s -> t -> t) -> c s -> t -> t myFold f cs = Endo ~~ myCrush (Endo . f) cs
What are we doing here? We're specifying the manner in which we use a structure-exploiting operator. We transform a function in a particular way by wrapping the range of that function in the type which indicates the structure to be exploited by the operator. We can capture this pattern by writing a transformer-transformer, composing the original function with a given packer, then composing the transformed function with the determined unpacker.
wise :: Unpack dp du => (bu -> bp) -> ((a -> bp) -> c -> dp) -> (a -> bu) -> c -> du wise way changes how = unpack . changes (way . how)
Third-order programming: it's a whole other order. So, now we have hints to the appropriate structure as parenthetical remarks:
travMap :: Traversable f => (s -> t) -> f s -> f t travMap = (Id `wise`) traverse
crush :: (Monoid m, Traversable f) => (s -> m) -> f s -> m crush = (Const `wise`) traverse
travFold :: (Traversable f) => (a -> b -> b) -> f a -> b -> b travFold = (Endo `wise`) crush
And, to boot,
instance Unpack Any Bool where unpack (Any b) = b
travExists :: (Traversable f) => (a -> Bool) -> f a -> Bool travExists = (Any `wise`) crush
instance Unpack All Bool where unpack (All b) = b
travAll :: (Traversable f) => (a -> Bool) -> f a -> Bool travAll = (All `wise`) crush
instance Unpack (Sum x) x where unpack (Sum x) = x
travSum :: (Num a, Traversable f) => f a -> a travSum = (Sum `wise`) crush id
I'm in the mood for some pointwise lifting (in pointfree style).
instance Unpack tp tu => Unpack (s -> tp) (s -> tu) where unpack = (unpack .)
Now we can exploit the pointwise lifting of |Unpack| together with the pointwise lifting of |Monoid|, yielding |travExists2|, a function which checks out whether any pair of elements drawn respectively from two traversable structures satisfies a relation. For example, |travExists2 (==)| will check if the two structures have an element in common.
travExists2 :: (Traversable f, Traversable g) => (a -> b -> Bool) -> f a -> g b -> Bool travExists2 = ((crush . (Any .)) `wise`) crush
How does it work? Well, we lift |r :: a -> b -> Bool| to |(Any .) . r :: a -> b -> Any| to |crush . (Any .) . r :: a -> g b -> Any|. Now, |g b -> Any| is a |Monoid| because Any is, so the outer crush lifts us to an |f a -> g b -> Any|. Then the unpacker for |g b -> Any| takes the returned function back to a |g b -> Bool|. All of this goes to show that Haskell has taken us far beyond the standard, unimaginative view of types that you see all over the place. I mean the view 'We already know what the programs are and how to run them; types just discriminate between good programs and bad programs.' That's an appropriate view for a typed /core/ language, but it misses so many great opportunities for a typed /programming/ language. We program by exposing structure. More to the point, locally, if we accept that some notion of type-level function is here to stay, is it worth adopting Unpack as a uniform way to work with those newtypes whose purpose is to indicate structure (as opposed to those which are intended to encapsulate invariants)? I don't know how wise 'wise' is, but I had fun playing with it. Unpack, though, is something I makes heavy use of. Thought I'd punt it out there... This message has been checked for viruses but the contents of an attachment may still contain software viruses, which could damage your computer system: you are advised to perform your own checks. Email communications with the University of Nottingham may be monitored as permitted by UK legislation.

On Mon, Oct 23, 2006 at 11:27:58AM +0100, Conor McBride wrote:
John Meacham wrote:
Although, now that we lost the Monoid instance for functions (which is very dismaying), it is less useful, as the monoid functions were very useful to build things up efficiently with (String -> String) as a type.
...(with which I rather disagree) to finish a note about programming with structure-indicating newtypes, like Endo.
I thought I'd also remind you of a little of what we bought by wrapping the endofunction monoid, then making the function instance do pointwise lifting instead.
I don't have an issue with the reasoning behind it. But the transition caused a lot of real and very subtle bugs that could have been avoided. the problem is that there are many places where the change caused silent changes to the behavior of existing code and these bugs were very hard to root out. for instance 6.4: mconcat [('a':),('b':),('c':),('d':)] [] => "abcd" and 6.6: mconcat [('a':),('b':),('c':),('d':)] [] => "abcd" but change it slightly: 6.4: mconcat [('a':),('b':),('c':),('d':)] "foo" => "abcdfoo" 6.6: mconcat [('a':),('b':),('c':),('d':)] "foo" => "afoobfoocfoodfoo" a very subtle change when buried deep in your typechecking code, can manifest in very strange and perplexing ways. you can't even search for every use of 'mconcat' to find them, as a lot of my uses were hidden inside of execWriter, or many other functions that internally use monoids. If someone on IRC didn't have the good idea of creating a dummy conflicting instance, I would probably have been finding such bugs for months to come... also, all the data constructors created by Data.Monoid are nice, but they are very common words and conflict with many pre-existing data types, which is bad for something as commonly imported as Data.Monoid. It would have been nicer to include them in somehing like Data.Monoid.Strategies. So, my beef wasn't so much with the reasoning behind the changes, but the execution of them. John -- John Meacham - ⑆repetae.net⑆john⑈

On 10/23/06, John Meacham
On Mon, Oct 23, 2006 at 11:27:58AM +0100, Conor McBride wrote:
John Meacham wrote:
Although, now that we lost the Monoid instance for functions (which is very dismaying), it is less useful, as the monoid functions were very useful to build things up efficiently with (String -> String) as a type.
...(with which I rather disagree) to finish a note about programming with structure-indicating newtypes, like Endo.
I thought I'd also remind you of a little of what we bought by wrapping the endofunction monoid, then making the function instance do pointwise lifting instead.
I don't have an issue with the reasoning behind it. But the transition caused a lot of real and very subtle bugs that could have been avoided.
the problem is that there are many places where the change caused silent changes to the behavior of existing code and these bugs were very hard to root out.
So, basically, you think we should have gone with the idea of putting both instances in newtypes for a while? (Not that I was around at the time or anything.)

On 10/22/06, John Meacham
Although, now that we lost the Monoid instance for functions (which is very dismaying), it is less useful, as the monoid functions were very useful to build things up efficiently with (String -> String) as a type.
WHAT!!!! Who, when, where, why, how????

On Mon, Oct 23, 2006 at 01:15:46PM -0400, Samuel Bronson wrote:
On 10/22/06, John Meacham
wrote: Although, now that we lost the Monoid instance for functions (which is very dismaying), it is less useful, as the monoid functions were very useful to build things up efficiently with (String -> String) as a type.
WHAT!!!!
Who, when, where, why, how????
The function composition instance is now wrapped in a newtype. The instances available now are listed here: http://www.haskell.org/ghc/docs/latest/html/libraries/base/Data-Monoid.html#... I was who; here is some of the rest: http://www.haskell.org/pipermail/libraries/2005-September/thread.html#4365

Ross Paterson wrote:
On Mon, Oct 23, 2006 at 01:15:46PM -0400, Samuel Bronson wrote:
On 10/22/06, John Meacham
wrote: Although, now that we lost the Monoid instance for functions (which is very dismaying), it is less useful, as the monoid functions were very useful to build things up efficiently with (String -> String) as a type.
WHAT!!!!
Who, when, where, why, how????
Funnily enough, that was my reaction when I saw that endofunctions under composition /was/ the library instance of Monoid for functions, thus preventing the powerful 'pointwise lifting' instance...
The function composition instance is now wrapped in a newtype. The instances available now are listed here:
http://www.haskell.org/ghc/docs/latest/html/libraries/base/Data-Monoid.html#...
I was who; here is some of the rest:
http://www.haskell.org/pipermail/libraries/2005-September/thread.html#4365
...which we now have instead, allowing things like newtype Parser a = Parser (String -> [(a, String]) deriving Monoid and plenty of other goodies. I have code (part of the Epigram layout library) which systematically iterates pointwise lifting to jack up operators on text boxes to operators on prioritised streams of text boxes, monoidal with respect to a lossy merge operation which discards obviously bad layouts and prevents combinatorial explosion. Pointwise lifting is one way to keep the programs simple even when the underlying structures become complex. While we're at it, would anybody object to a few other instances of the same idea? for <blah> in {State s, IO, Id}, may we have suitable Applicative instances where currently absent, then
instance Monoid x => Monoid (<blah> x) where mempty = pure mempty mappend sx sy = pure mappend <*> sx <*> sy
? Basically, I'd suggest that any monad/idiom/whatever should lift monoids if it doesn't induce a monoid structure of its own (as [], Maybe, etc) do. What to do with StateT etc is more complex... Does this make sense? Cheers Conor This message has been checked for viruses but the contents of an attachment may still contain software viruses, which could damage your computer system: you are advised to perform your own checks. Email communications with the University of Nottingham may be monitored as permitted by UK legislation.

Donald Bruce Stewart schrieb:
* intercat * the problem with joinWith (or whatever) is that traditionally it would be joinBy
As I pointed out in http://thread.gmane.org/gmane.comp.lang.haskell.glasgow.user/10136/focus=136... the "By" suffix characterizes functions taking _binary_ comparison arguments. So I only see the tradition of stating this to be a problem. Cheers Christian

On 10/23/06, Christian Maeder
Donald Bruce Stewart schrieb:
* intercat * the problem with joinWith (or whatever) is that traditionally it would be joinBy
As I pointed out in http://thread.gmane.org/gmane.comp.lang.haskell.glasgow.user/10136/focus=136... the "By" suffix characterizes functions taking _binary_ comparison arguments. So I only see the tradition of stating this to be a problem.
I agree. A join* name would be nice but there doesn't seem to be a nice suffix that doesn't give the wrong connotation. So I'm sticking with 'intercalate'. But I'll wait until all the discussion have settled until I apply the patch. All the best, Josef

Hello,
I am a bit confused about the process of modifying the core
libraries... Just the other day there was a suggested process that
patches are sent to the list and discussed, etc. which happened with
|intercalate| and clearly there is some controversy about its
usefulness but it seems that now it has been committed to the library
anyways. It seems that the process is well-defined in the extreme
cases (only +ve or only -ve feedback) and for indifference (no
feedback=accept)
but what should we do in situations such as this one, where there is
some +ve and some -ve feedback? Perhaps by symmetry
mixed-feedback=reject :-) (this last proposal is not entirely serious)
-Iavor
On 10/27/06, Josef Svenningsson
On 10/23/06, Josef Svenningsson
wrote: So I'm sticking with 'intercalate'. But I'll wait until all the discussion have settled until I apply the patch.
Just so you all know. I've applied the patch now.
All the best,
Josef _______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

josef.svenningsson:
On 10/23/06, Josef Svenningsson
wrote: So I'm sticking with 'intercalate'. But I'll wait until all the discussion have settled until I apply the patch.
Just so you all know. I've applied the patch now.
All the best,
Josef
Josef, Would it be possible to create a Trac ticket summarising the discussion that occured, following the guide here: http://haskell.org/haskellwiki/Library_submissions In order to satisfy everyone that there was a consensus, in this case? -- Don

On 10/28/06, Donald Bruce Stewart
josef.svenningsson:
Just so you all know. I've applied the patch now.
Josef,
Would it be possible to create a Trac ticket summarising the discussion that occured, following the guide here: http://haskell.org/haskellwiki/Library_submissions
Done. It's ticket #971: http://hackage.haskell.org/trac/ghc/ticket/971#preview
In order to satisfy everyone that there was a consensus, in this case?
Right. I'm sorry if anyone feels that I've rushed this patch. To me it seemed to be a clear majority in favour but maybe I should have given a heads up before applying the patch. All the best, Josef

On 10/28/06, Josef Svenningsson
On 10/28/06, Donald Bruce Stewart
wrote: josef.svenningsson:
Just so you all know. I've applied the patch now.
Josef,
Would it be possible to create a Trac ticket summarising the discussion that occured, following the guide here: http://haskell.org/haskellwiki/Library_submissions
Done. It's ticket #971: http://hackage.haskell.org/trac/ghc/ticket/971#preview
Given the feedback I got from this I've decided to pull split out of the patch. It will only concern intercalate now.
In order to satisfy everyone that there was a consensus, in this case?
Right. I'm sorry if anyone feels that I've rushed this patch. To me it seemed to be a clear majority in favour but maybe I should have given a heads up before applying the patch.
I did apply the patch much to soon and I don't have a good excuse for it. I've rolled it back and uploaded the new patches to trac. I've also set a deadline for the patch now. Since we've already discussed it quite a lot I've set a pretty tight deadline: Monday Nov 6. All this information can be found in the ticket which I linked to above. All the best, Josef Svenningsson

On Sun, Oct 22, 2006 at 12:50:14PM +1000, Donald Bruce Stewart wrote:
Getting bogged down in fiddly details will just derail this effort. See here. Perfectionists will not be tolerated! ;) http://haskell.org/haskellwiki/Protect_the_community [...] So, I propose we start with just, say, 'connect', with some QC properties, and get that into base, before we get dragged out into a big discussion about the entire api design.
I don't care much about this particular case, but we really need a better way of handling interface changes than this. We don't want proposals to rot, but changes to basic interfaces also need thorough consideration. At present, unless a proposal meets with a chorus of approval, the only way to get a decision is from SimonM or unilateral action by some committer. That needs to change, I think. BTW, I find that "Protecting the community from poisonous people" stuff incongruously negative compared with my experiences on the Haskell mailing lists. There are some positive suggestions in there; let's focus on those, rather than the witch hunting.

ross:
On Sun, Oct 22, 2006 at 12:50:14PM +1000, Donald Bruce Stewart wrote:
Getting bogged down in fiddly details will just derail this effort. See here. Perfectionists will not be tolerated! ;) http://haskell.org/haskellwiki/Protect_the_community [...] So, I propose we start with just, say, 'connect', with some QC properties, and get that into base, before we get dragged out into a big discussion about the entire api design.
I don't care much about this particular case, but we really need a better way of handling interface changes than this. We don't want proposals to rot, but changes to basic interfaces also need thorough consideration. At present, unless a proposal meets with a chorus of approval, the only way to get a decision is from SimonM or unilateral action by some committer. That needs to change, I think.
Agreed. At least a method by which proposals get Trac tickets and then due consideration (though avoiding voting would be nice). Any suggestions?
BTW, I find that "Protecting the community from poisonous people" stuff incongruously negative compared with my experiences on the Haskell mailing lists. There are some positive suggestions in there; let's focus on those, rather than the witch hunting.
Oh yes, it's nothing to do with the Haskell community at all! These were notes from the Subversion community, that I took down at the Google Summit last week. The Haskell community seems to function a _lot_ better :) There are some useful hints there though. -- Don

Hi
At least a method by which proposals get Trac tickets and then due consideration (though avoiding voting would be nice).
I disagree - maybe avoid voting but at the very least require n supporting people from the Haskell community - at least some way to see who agrees and who disagrees. There are lots of totally ridiculous proposals that go out on the Haskell' mailing list, which I (and most people) ignore. I'd hate for a silly library proposal to sneak in because no one realised anyone was formally considering it! I'd also demand GHC + Hugs + <one other> compiler support, plus Windows + Linux + Mac support. I'd even go as far as to remove the evil API's that have snuck in which don't meet these criterial. Thanks Neil

ndmitchell:
Hi
At least a method by which proposals get Trac tickets and then due consideration (though avoiding voting would be nice).
I disagree - maybe avoid voting but at the very least require n supporting people from the Haskell community - at least some way to see who agrees and who disagrees. There are lots of totally ridiculous proposals that go out on the Haskell' mailing list, which I (and most people) ignore. I'd hate for a silly library proposal to sneak in because no one realised anyone was formally considering it!
I'd also demand GHC + Hugs + <one other> compiler support, plus Windows + Linux + Mac support. I'd even go as far as to remove the evil API's that have snuck in which don't meet these criterial.
Yes, Perhaps if we have some requirement that proposals are considered only if they come with code (darcs patches), QuickCheck properties, portablity checks, this would keep the signal /noise ratio higher, make it easier to consider and move on proposals, and make any consensus easier to reach. Maybe it really is time to try to formalise a process for further library growth. -- Don

On Mon, Oct 23, 2006 at 09:38:38PM +1000, Donald Bruce Stewart wrote:
ross:
I don't care much about this particular case, but we really need a better way of handling interface changes than this. We don't want proposals to rot, but changes to basic interfaces also need thorough consideration. At present, unless a proposal meets with a chorus of approval, the only way to get a decision is from SimonM or unilateral action by some committer. That needs to change, I think.
Agreed.
At least a method by which proposals get Trac tickets and then due consideration (though avoiding voting would be nice).
Any suggestions?
I think the most important things are: - the proposed change is expressed as a darcs patch that compiles, including Haddock documentation that compiles, and possibly tests if appropriate. (This ensures that there is something concrete to discuss.) - the proposed change gets a ticket and a timescale for consideration. Active tickets are publicized, and silence during this period can be taken as consent. (The timescale is necessary to force and focus a discussion, and also to bound the effort required.) - at the end of the discussion period, the proposer adds to the ticket a summary of the relevant parts of the discussion, the ticket is archived and, if consensus was achieved, the change is made. (The summary is needed for anyone wondering about the change later: it's not reasonable to point people at a 50-message thread.) A deeply held disagreement at the end of the discussion period may require some form of government (voting, dictatorship, etc), but I think that will be fairly rare.

On 10/24/06, Ross Paterson
- the proposed change is expressed as a darcs patch that compiles, including Haddock documentation that compiles, and possibly tests if appropriate. (This ensures that there is something concrete to discuss.)
Btw, haddock documentation doesn't compile for me. (My haddock doesn't like the --source-module flag, and there doesn't seem to be a newer one in Debian testing.)

Am Dienstag, 24. Oktober 2006 17:30 schrieb Samuel Bronson:
Btw, haddock documentation doesn't compile for me. (My haddock doesn't like the --source-module flag, and there doesn't seem to be a newer one in Debian testing.)
There was a new Haddock 0.8 release a few weeks ago, and on http://haskell.org/haddock/ FreeBSD/x86 is mentioned explicitly. I don't have any Debian around, so I can't test this, but is there anything wrong with the recipe on the Haddock page? If yes, I (or SimonM) will happily change it. Cheers, S.

Just a quick suggestion. My experience is that Formal consensus works very well in situations where informal consensus (what you guys seem to be currently using) isn't powerful enough. I've found this resource to be a good one: http://www.consensus.net/ocaccontents.html Many larger organisations use such formal mechanisms to achieve swift consensus and avoid brute majoritarian democracy or "benevolent" (if you're lucky!) dictators. If people are interested, I'd be happy to share my understanding of the matter and help facilitate such a system. On Mon, 2006-10-23 at 12:16 +0100, Ross Paterson wrote:
On Sun, Oct 22, 2006 at 12:50:14PM +1000, Donald Bruce Stewart wrote:
Getting bogged down in fiddly details will just derail this effort. See here. Perfectionists will not be tolerated! ;) http://haskell.org/haskellwiki/Protect_the_community [...]
I don't care much about this particular case, but we really need a better way of handling interface changes than this. We don't want proposals to rot, but changes to basic interfaces also need thorough consideration. At present, unless a proposal meets with a chorus of approval, the only way to get a decision is from SimonM or unilateral action by some committer. That needs to change, I think. -- Robert Marlow

dons@cse.unsw.edu.au (Donald Bruce Stewart) writes:
http://www.google.com/codesearch?q=file%3A%5C.hs+intersperse&btnG=Search+Code
Oh, that's a nice trick for working out apis. Good idea!
(And don't forget there's also: http://www.google.com/codesearch?q=file%3A%5C.lhs+intersperse&btnG=Search+Code ) -k -- If I haven't seen further, it is by standing in the footprints of giants

By the way - about Google's Code Search - they only seem to support SVN and CVS - anybody else must provide tgz-files or similar? Who do we have to bribe to get them to accept darcs archives -- which, mind you, only requires them to scan the files in a regular directory (perhaps *avoiding* the _darcs directory). -k -- If I haven't seen further, it is by standing in the footprints of giants

On Sat, Oct 21, 2006 at 03:29:57PM +0200, Josef Svenningsson wrote:
Hi,
I'd like to propose to include the following function in Data.List: join :: [a] -> [[a]] -> [a] join x xs = concat (intersperse x xs)
I find that every time I use intersperse I also concat the result. And it seems that I'm not alone: http://www.google.com/codesearch?q=file%3A%5C.hs+intersperse&btnG=Search+Code A clear majority of all the uses of intersperse also use concat on the result. I think it seems worthwhile to give a name to that idiom.
The name 'join' was taken from Data.ByteString, where a similar function exists.
All the best,
I have that same function in my standard toolbox, but call it concatInter a la concatMap. though mconcatInter would probably be better, since then we can do it for an arbitrary monoid. John -- John Meacham - ⑆repetae.net⑆john⑈
participants (29)
-
Aaron Denney
-
Bulat Ziganshin
-
Cale Gibbard
-
Christian Maeder
-
Clifford Beshers
-
Conor McBride
-
dons@cse.unsw.edu.au
-
Henning Thielemann
-
Iavor Diatchki
-
isto
-
Jason Dagit
-
Jeremy Shaw
-
John Meacham
-
Jon Fairbairn
-
Josef Svenningsson
-
Jón Fairbairn
-
kahl@cas.mcmaster.ca
-
Ketil Malde
-
Krasimir Angelov
-
Malcolm Wallace
-
Neil Mitchell
-
Robert Dockins
-
Robert Marlow
-
Ross Paterson
-
Samuel Bronson
-
Simon Marlow
-
Sven Panne
-
Tomasz Zielonka
-
Udo Stenzel