Arguments against an hypothetical Data.ByteString.Generic?

Hello, As vector provides a class generalizing all flavours (Data.Vector.Generic.Vector), it occurs to me that the same could be done for ByteString. Then, packages based on it would have the choice between hardcoded and generic, they wouldn't have to duplicate a module to handle both strict and lazy versions, as (with the exception of functions for communication with C code) they already provide the same API. I would be willing to make it, it's a concern I've had in mind for a long time, but as I'm pretty sure the idea isn't new, I would very much like to know if and what arguments (related to performance maybe ? I don't know...) were raised against that.

On 27 March 2012 11:00, Yves Parès
Hello,
As vector provides a class generalizing all flavours (Data.Vector.Generic.Vector), it occurs to me that the same could be done for ByteString. Then, packages based on it would have the choice between hardcoded and generic, they wouldn't have to duplicate a module to handle both strict and lazy versions, as (with the exception of functions for communication with C code) they already provide the same API. I would be willing to make it, it's a concern I've had in mind for a long time, but as I'm pretty sure the idea isn't new, I would very much like to know if and what arguments (related to performance maybe ? I don't know...) were raised against that.
It's not entirely what you need but are you aware of my vector-bytestring library? http://hackage.haskell.org/package/vector-bytestring It doesn't (yet) abstract over strict and lazy ByteStrings. But that would be a nice addition! In an ideal world we would have a Lazy type family which for each type of vector would return its lazy version (where the vector is unpacked in the cons cell). Then we would need your generic API for working with both lazy and strict vectors. Regards, Bas

Hi Bas, Yes, thank you to remind me of that, I remember now having seen the project. Strict ByteStrings being an alias to Vector Word8 is a good idea (are bytestrings are already implemented exactly like Data.Vector.Storable.Vector). But in that case we could use the API of vector for bytestrings (the bytestring API would be provided only for backwards compatibility, right?). Does vector-bytestring plans to be the new implementation for bytestrings in the end or is it a side-package?
In an ideal world we would have a Lazy type family which for each type of vector would return its lazy version
What about a type like:
data Vector v a = Empty | Chuck {-# UNPACK #-} !(v a) (Vector v a)
??
GHC accepts it.
And then every function for lazy vectors can then be written like:
import qualified Data.Vector.Generic as G (Vector(..))
cons :: (G.Vector v) => a -> Vector v a -> Vector v a
cons x v = ...
and also (even better):
instance (G.Vector v) => G.Vector (Vector v) where
cons x v = ...
Just make aliases to follow the API, for instance in Data.Vector.Lazy:
import qualified Data.Vector.Lazy.Internal as L
import qualified Data.Vector as V
type Vector a = L.Vector V.Vector a
Le 27 mars 2012 20:38, Bas van Dijk
On 27 March 2012 11:00, Yves Parès
wrote: Hello,
As vector provides a class generalizing all flavours (Data.Vector.Generic.Vector), it occurs to me that the same could be done for ByteString. Then, packages based on it would have the choice between hardcoded and generic, they wouldn't have to duplicate a module to handle both strict and lazy versions, as (with the exception of functions for communication with C code) they already provide the same API. I would be willing to make it, it's a concern I've had in mind for a long time, but as I'm pretty sure the idea isn't new, I would very much like to know if and what arguments (related to performance maybe ? I don't know...) were raised against that.
It's not entirely what you need but are you aware of my vector-bytestring library?
http://hackage.haskell.org/package/vector-bytestring
It doesn't (yet) abstract over strict and lazy ByteStrings. But that would be a nice addition!
In an ideal world we would have a Lazy type family which for each type of vector would return its lazy version (where the vector is unpacked in the cons cell). Then we would need your generic API for working with both lazy and strict vectors.
Regards,
Bas

On 27 March 2012 21:46, Yves Parès
Yes, thank you to remind me of that, I remember now having seen the project. Strict ByteStrings being an alias to Vector Word8 is a good idea (are bytestrings are already implemented exactly like Data.Vector.Storable.Vector). But in that case we could use the API of vector for bytestrings (the bytestring API would be provided only for backwards compatibility, right?).
Yes, I hope that one day the bytestring package and the ByteString type will be deprecated in favor of vector and Data.Vector.Storable.Vector Word8 respectively. vector-bytestring is indeed intended as a package which should make the transition easier.
Does vector-bytestring plans to be the new implementation for bytestrings in the end or is it a side-package?
I hope that once we get on par with bytestring's performance we can replace it (we're almost there!, Yell if you want to see some benchmark results).
In an ideal world we would have a Lazy type family which for each type of vector would return its lazy version
What about a type like:
data Vector v a = Empty | Chuck {-# UNPACK #-} !(v a) (Vector v a) ??
If you build with -Wall you'll see the following unfortunate warning: Warning: Ignoring unusable UNPACK pragma on the first argument of `Chunk' Johan Tibell recently discussed some of his ideas on how to solve this: http://www.haskell.org/pipermail/glasgow-haskell-users/2012-March/022079.htm... But for now we need to make a specialized type for every different vector type and use an associated type family to abstract over these different types. Regards, Bas

Oh, okay ^^ sorry, my bad. I should have compiled with -Wall.
So I started a repo at https://github.com/YwenP/bytestring-generic
Le 27 mars 2012 22:01, Bas van Dijk
Yes, thank you to remind me of that, I remember now having seen the
On 27 March 2012 21:46, Yves Parès
wrote: project. Strict ByteStrings being an alias to Vector Word8 is a good idea (are bytestrings are already implemented exactly like Data.Vector.Storable.Vector). But in that case we could use the API of vector for bytestrings (the bytestring API would be provided only for backwards compatibility, right?).
Yes, I hope that one day the bytestring package and the ByteString type will be deprecated in favor of vector and Data.Vector.Storable.Vector Word8 respectively. vector-bytestring is indeed intended as a package which should make the transition easier.
Does vector-bytestring plans to be the new implementation for bytestrings in the end or is it a side-package?
I hope that once we get on par with bytestring's performance we can replace it (we're almost there!, Yell if you want to see some benchmark results).
In an ideal world we would have a Lazy type family which for each type of vector would return its lazy version
What about a type like:
data Vector v a = Empty | Chuck {-# UNPACK #-} !(v a) (Vector v a) ??
If you build with -Wall you'll see the following unfortunate warning:
Warning: Ignoring unusable UNPACK pragma on the first argument of `Chunk'
Johan Tibell recently discussed some of his ideas on how to solve this:
http://www.haskell.org/pipermail/glasgow-haskell-users/2012-March/022079.htm...
But for now we need to make a specialized type for every different vector type and use an associated type family to abstract over these different types.
Regards,
Bas
participants (2)
-
Bas van Dijk
-
Yves Parès