Code review: list-like datatype with constant time appending based on function composition.

Hello everybody I would kindly ask you to take a look at this code and tell me your opinions: https://gist.github.com/nicola-gigante/43533ce907f88a6aba16 https://gist.github.com/nicola-gigante/43533ce907f88a6aba16 The file try to implement a list-like data type, based on function composition to achieve a constant-time appending operation (in contrast to linear (++)). I’m sure this is something old. If you have material that covers an approach like this, please tell me. I’ve tried to implement all the type classes that I currently use, and it seems to work. Also, all the operations of those typeclasses seems to preserve the original laziness properties, despite I was expecting the contrary. A list in this code is basically a function that’s constructed by composing a lot of partially applied cons operators. To obtain a plain list from it, it’s sufficient to apply the empty list at the end. So I was expecting that to evaluate even the first element of the list, the code would have to evaluate all the function composition chain to obtain the entire function to being able to apply the final argument. It seems not true, since this code can still manipulate infinite lists. For example:
fmap (*2) . fromList $ repeat 42
This code prints an infinite stream of “84” numbers in GHCI. It doesn’t wait to have the entire list to then print it out. In my understanding that means I’m building the list sufficiently lazily, or am I missing something? In the case this actually means my datatype is lazy enough, why isn’t it more strict instead, following the above reasoning? Is it the case that haskell laziness extends to partially constructed functions? Also, the implementation is very simple. Basically I use the list functions and go back and forth with toList/fromList. It all seems too easy… am I wasting a lot of time or space somewhere? Thank you a lot for your help, Greetings, Nicola

On Tue, Nov 18, 2014 at 04:35:34PM +0100, Nicola Gigante wrote:
I would kindly ask you to take a look at this code and tell me your opinions:
https://gist.github.com/nicola-gigante/43533ce907f88a6aba16 https://gist.github.com/nicola-gigante/43533ce907f88a6aba16 [..] I’m sure this is something old. If you have material that covers an approach like this, please tell me.
It seems that you have discovered the "DList". http://hackage.haskell.org/package/dlist-0.7.1/docs/Data-DList.html

(...) The file try to implement a list-like data type, based on function composition to achieve a constant-time appending operation (in contrast to linear (++)).
I’m sure this is something old. If you have material that covers an approach like this, please tell me.
There is this paper of ICLP 1991 (even though the implementation language is lambda-Prolog): "Naive Reverse can be Linear", by Brisset P. and Ridoux O. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.24.8975 Which itself, references this paper of 1986 Information Processing Letters (n° 22, pp. 141-144): "A novel representationof lists and its application to the function “reverse”", by Hugues R.J.M. http://www.cs.tufts.edu/~nr/cs257/archive/john-hughes/lists.pdf Regards. --Serge

Il giorno 18/nov/2014, alle ore 16:59, Serge Le Huitouze
ha scritto: (...) The file try to implement a list-like data type, based on function composition to achieve a constant-time appending operation (in contrast to linear (++)).
I’m sure this is something old. If you have material that covers an approach like this, please tell me.
There is this paper of ICLP 1991 (even though the implementation language is lambda-Prolog): "Naive Reverse can be Linear", by Brisset P. and Ridoux O. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.24.8975
Which itself, references this paper of 1986 Information Processing Letters (n° 22, pp. 141-144): "A novel representationof lists and its application to the function “reverse”", by Hugues R.J.M. http://www.cs.tufts.edu/~nr/cs257/archive/john-hughes/lists.pdf
Thank you Serge and Tom for your quick answers! I’m surely going to read those paper, and use Data.DList instead of my own implementation, to not reinvent the wheel. What about my doubts on why it works so lazily?
Regards.
—Serge
Greetings, Nicola

What about my doubts on why it works so lazily?
Suppose 'f' is the function '(1 :)', and 'g' is some other unspecified other function of type '[Integer] -> [Integer]' ("difference list"). You may evaluate the concatenation '(f . g) []' like this: (f . g) [] = f (g []) = 1 : g [] and at this point, you already have partial knowledge of the resulting list. Note that we use "lazy evaluation", in the sense that we do not evaluate the argument 'g []' to the function 'f' right away, but proceed by first substituting the definition of 'f'.

Il giorno 18/nov/2014, alle ore 17:37, Arie Peterson
What about my doubts on why it works so lazily?
Suppose 'f' is the function '(1 :)', and 'g' is some other unspecified other function of type '[Integer] -> [Integer]' ("difference list").
You may evaluate the concatenation '(f . g) []' like this:
(f . g) [] = f (g []) = 1 : g []
and at this point, you already have partial knowledge of the resulting list.
Note that we use "lazy evaluation", in the sense that we do not evaluate the argument 'g []' to the function 'f' right away, but proceed by first substituting the definition of 'f’.
Thanks, that makes sense, but it still seem to me that it depends on the order of application of the composition operator. For example, what if the structure has been constructed by a left fold, so I have: ((((((f . g) . h) …..) In this case, the evaluation have to descend ’n’ thunks to find the first function to apply, so I can’t handle infinite lists. On the other hand, I can’t handle such an infinite list from a left fold anyway, so maybe that’s the point? Speaking about this “difference list” more generally, which are the negative sides of using such a data structure? For example, when should I use Data.Sequence, for example, that requires Ord on my data types, while DList offers more or less the same features (or am I missing something?) Performance? Thank you very much, Nicola

You might also be intrested in my and oleg's paper "reflection without
remorse" which discusses this construction in a more general setting, the
problems with it and their solution
Paper: http://homepages.cwi.nl/~ploeg/papers/zseq.pdf
Talk: https://www.youtube.com/watch?v=_XoI65Rxmss
On Nov 18, 2014 5:38 PM, "Arie Peterson"
What about my doubts on why it works so lazily?
Suppose 'f' is the function '(1 :)', and 'g' is some other unspecified other function of type '[Integer] -> [Integer]' ("difference list").
You may evaluate the concatenation '(f . g) []' like this:
(f . g) [] = f (g []) = 1 : g []
and at this point, you already have partial knowledge of the resulting list.
Note that we use "lazy evaluation", in the sense that we do not evaluate the argument 'g []' to the function 'f' right away, but proceed by first substituting the definition of 'f'.
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

On Tue, Nov 18, 2014 at 05:07:21PM +0100, Nicola Gigante wrote:
I’m surely going to read those paper, and use Data.DList instead of my own implementation, to not reinvent the wheel.
What about my doubts on why it works so lazily?
I don't know why you'd be doubtful about laziness. It seems perfectly lazy to me. I wrote the post a while ago on how DList works. Perhaps it will help you. http://h2.jaguarpaw.co.uk/posts/demystifying-dlist/ Tom

Hi Tom, Your "Tree" construction does generalize to monads. Also see my paper and oleg's paper "reflection without remorse" referenced above. Cheers, Atze On Nov 19, 2014 5:46 AM, "Tom Ellis" < tom-lists-haskell-cafe-2013@jaguarpaw.co.uk> wrote:
On Tue, Nov 18, 2014 at 05:07:21PM +0100, Nicola Gigante wrote:
I’m surely going to read those paper, and use Data.DList instead of my own implementation, to not reinvent the wheel.
What about my doubts on why it works so lazily?
I don't know why you'd be doubtful about laziness. It seems perfectly lazy to me. I wrote the post a while ago on how DList works. Perhaps it will help you.
http://h2.jaguarpaw.co.uk/posts/demystifying-dlist/
Tom _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
participants (5)
-
Arie Peterson
-
Atze van der Ploeg
-
Nicola Gigante
-
Serge Le Huitouze
-
Tom Ellis