(2) the York Binary library uses the IO monad, and presumably various variables within a BinHandle, to keep track of state. I think this is unnecessary, for example I don't think the process of converting a value to a byte array should really have to go through IO. We are supposed to be functional programmers after all.
When writing to a BinMem it would be possible to avoid the IO monad, and use something like the ST monad instead. However, I'm not sure it's worth making this generalisation, since it would necessarily complicate the library quite a bit. Do you have an application where being in the IO monad to do this stuff isn't possible, or is inconvenient? Cheers, Simon
On Wed, May 21, 2003 at 11:52:00AM +0100, Simon Marlow wrote:
When writing to a BinMem it would be possible to avoid the IO monad, and use something like the ST monad instead. However, I'm not sure it's worth making this generalisation, since it would necessarily complicate the library quite a bit.
Do you have an application where being in the IO monad to do this stuff isn't possible, or is inconvenient?
Since unboxed arrays have no outgoing edges in the heap graph, I can think of using such library to reduce the heap graph (thus reducing GC time) by storing some data in memory in binary form. Limiting library's interface to IO monad would make it hard to use such technique in purely functional code. Maybe GHC could itself compress live but rarely accessed data in the heap? Regards, Tom -- .signature: Too many levels of symbolic links
Simon Marlow wrote:
Do you have an application where being in the IO monad to do this stuff isn't possible, or is inconvenient? No. But it is inelegant, since IO is the *only* way the bit consumer has of preserving state. So if for example the target is a list of arrays and bytes, it would be necessary to hold the list inside an IORef or something similar.
Do you have an application where being in the IO monad to do this stuff isn't possible, or is inconvenient? No. But it is inelegant, since IO is the *only* way the bit consumer has of preserving state. So if for example the target is a list of arrays and bytes, it would be necessary to hold the list inside an IORef or something similar.
You lost me here :). I can easily agree with "it is inelegant", but I think that as soon as you start talking about binary representations of data types, you have to give up some elegance. I don't understand your example of the bit consumer and IORefs though...could you explain it a bit more (no pun)?
Hal Daume III wrote:>>>Do you have an application where being in the IO monad to do this stuff
isn't possible, or is inconvenient?
No. But it is inelegant, since IO is the *only* way the bit consumer has of preserving state. So if for example the target is a list of arrays and bytes, it would be necessary to hold the list inside an IORef or something similar.
You lost me here :). I can easily agree with "it is inelegant", but I think that as soon as you start talking about binary representations of data types, you have to give up some elegance.
I don't understand your example of the bit consumer and IORefs though...could you explain it a bit more (no pun)?
See the message I just posted the libraries list for a better example. http://haskell.org/pipermail/libraries/2003-May/001006.html Another one would be where you want to read some data from an untrusted client, limiting the number of bytes you can be bothered to read. This could be done using my framework by extending the monad with a count, it can't be done in Malcolm's framework, at least not without introducing an extra first step to do the IO. Malcolm, I hope you'll forgive me for being so rude about your work. Standing on the shoulders of giants, and all that ... best wishes to all, George
Hi George,
I don't understand your example of the bit consumer and IORefs though...could you explain it a bit more (no pun)?
See the message I just posted the libraries list for a better example. http://haskell.org/pipermail/libraries/2003-May/001006.html
Why not use a state transformer on top of IO or something like that?
Another one would be where you want to read some data from an untrusted client, limiting the number of bytes you can be bothered to read. This could be done using my framework by extending the monad with a count, it can't be done in Malcolm's framework, at least not without introducing an extra first step to do the IO.
Hrm, this is interesting :). It seems like you should be able to use lazyGet to achieve this. Opening a BinIO doesn't actually cause it to be read until you actually do some of the reading. Using lazyGet should enable you to just read the amount you need to read under certain circumstances. I envision something like: newtype UntrustedString = UntrustedString String instance Binary UntrustedString where put_ bh (UntrustedString s) = put_ bh s -- put_ is trusted get bh = do s <- lazyGet bh let s' = deepSeq $ take 20 s s' `seq` closeHandle bh return (UntrustedString s') or something like that (untested, of course).
I wrote
See the message I just posted the libraries list for a better example. http://haskell.org/pipermail/libraries/2003-May/001006.html
to which Hal Daume wrote (snipped)
Why not use a state transformer on top of IO or something like that?
Another one would be where you want to read some data from an untrusted client, limiting the number of bytes you can be bothered to read. This could be done using my framework by extending the monad with a count, it can't be done in Malcolm's framework, at least not without introducing an extra first step to do the IO.
Hrm, this is interesting :). It seems like you should be able to use lazyGet to achieve this. Opening a BinIO doesn't actually cause it to be read until you actually do some of the reading. Using lazyGet should enable you to just read the amount you need to read under certain circumstances. I envision something like:
newtype UntrustedString = UntrustedString String
instance Binary UntrustedString where [..] That's not quite what I want, what I want is a way of reading a type with an existing binary representation, throwing an exception if you
Er, how exactly does a state transformer allow my implementation of put :: BinHandle -> a -> IO (BinPtr a) to get at the extra state? try to read too many characters. Of course it's possible with Malcolm's framework to define a set of parallel types to the standard Haskell ones (UntrustedInt, UntrustedString, and so on) and define new instances of Binary for the lot, but this would be painful in general. And you'd have to do it all over again should you want to put something else inbetween the individual read/write operations (logging, online compression, and so on) and the actual output. However I use the counting-bytes example because I implemented it a few weeks ago (writing server code, if you want to know), except that I did it inelegantly, before I thought of this new framework. As I said before, I have twice written inelegant converters to and from binary, in the project I am now working on. It occurs to me that in both cases I need some sort of extra state or information, meaning that in neither case would Malcolm's framework have been of use without hackery.
Er, how exactly does a state transformer allow my implementation of put :: BinHandle -> a -> IO (BinPtr a) to get at the extra state?
Sorry, wasn't thinking clearly. I guess what I had in the back of my mind was something like this: What if all the functions had, instead of type signature '... -> IO a', had signature 'MonadIO m => ... -> m a'? Presumably this could be done, essentially lifting all the IO operations into a monad transformer inside the Binary class. Then the Binary implementation is happy because it can use IO as it sees fit, and you're happy because you can piggy-back whatever you want on top of IO. This all goes back to SimonM's original questions as to whether there would be much to be gained from allowing BinMem operations in ST. If the answer is "not much", then the MonadIO solution seems reasonable.
That's not quite what I want, what I want is a way of reading a type with an existing binary representation, throwing an exception if you try to read too many characters. Of course it's possible with Malcolm's framework to define a set of parallel types to the standard Haskell ones (UntrustedInt, UntrustedString, and so on) and define new instances of Binary for the lot, but this would be painful in general. And you'd have to do it all over again should you want to put something else inbetween the individual read/write operations (logging, online compression, and so on) and the actual output.
The reason I used String is because it seems somewhat strange to define a maximum number of "characters" for an arbitrary type. It seems that the maximal amount to read should depend on the type you are reading; otherwise you might end up reading only enough to represent half of your data structure. I think perhaps the best you can do in this situation is just to read an array of word8s of some length and then let your own function do the checking. I understand the intent of only reading a certain amount from an untrusted source, but I'm still unclear as to how it actually works. - Hal
participants (4)
-
George Russell -
Hal Daume III -
Simon Marlow -
Tomasz Zielonka