Bulat Ziganshin wrote:
Wednesday, February 08, 2006, 2:58:30 PM, you wrote: SM> I would prefer to see more type structure, rather than putting SM> everything in the Stream class. You have classes ByteStream, SM> BlockStream etc, but these are just renamings of the Stream class. There SM> are many compositions that are illegal, but we don't find out until SM> runtime; it would make a lot more sense to me to expose this structure SM> in the type system.
i initially used normal splitted classes (vGetBuf was in BlockStream) and so on, but come accross problems with the type classes system and decided to simplify the design. now i feel himself more confident with the classes, feel that i know source of my previous problems and therefore slowly migrate back to the splitted classes design. the library as published is just on the half of this way. but i know some limitations. that is the one problem:
data BinHandle = forall h . (Stream IO h) => BinH h
One possibility is something like this: data BinHandle = forall h . (Stream IO h, Typeable h) => BinH h then you can recover the original stream type (by guessing what it is). Or there are other solutions - adding an extra field to BinH, or separating the BinH constructor into two, one with a MemoryStream and one without. What's interesting is that you're saying you really want dynamic typing here - you don't want to distinguish different types of BinHandle, instead you want the saveToFile operation to fail at runtime if the wrong kind of stream is used. It's slightly strange to use dynamic typing here when the rest of the library would be using static typing, so it might be worthwhile considering static typing solutions instead: don't use an existential here, just add h as a parameter of BinHandle. This does mean you have to add Stream predicates a lot of places, though. Someday (soon, I hope), GHC will let you say data BinHandle h where BinH :: Stream IO h => BinH h but it doesn't work right now (or at least, it doesn't do what you want).
moreover, splitting the Streams interface will require from the library users to give more classes in defining context for their functions, like the:
process :: (Stream IO h, Seekable IO h, Buffered h) => h -> IO ()
that is not so good, especially if adding new interfaces means slowdown of calls to this function
you can combine multiple classes with a dummy superclass. If we get class synonyms (see Haskell' proposal) this will get easier. Performance of the example above might actually be better than having a single Stream class, depending on how much dictionary *building* needs to happen. In your library, every time an overloaded Stream function is called, it must be passed a dictionary for Stream, which is a tuple with 20+ elements. These dictionaries will probably be built at runtime, because of the superclass structure (the compiler usually won't be able to predict what layering of stream transformers will be used, and hence what dictionaries will be needed). You can provide some specialisations to help - SPECIALISE INSTANCE should be useful here. The point is that the performance implications aren't obvious, it depends a lot on how much sharing of dictionaries happens.
SM> Also I'd like to see separate SM> input/output streams for even more type safety, and I believe SM> simplicity,
it will be great! but it is very uneasy and even seems impossible:
1) this will prevent dividing streams into the MemoryStream/BlockStream/ByteStream, what i like you consider as more important. it is impossible to say what InputStream BlockStream implements only vGetBuf, while OutputStream BlockStream implements only vPutBuf operation
I don't think so - you just have InputByteStream/OutputByteStream classes, and similarly for the others.
2) such division will require to implement 2 or 3 (+ReadWrite) times more Stream types than now. Say, instead of FD we will get InputFD and OutputFD, instead of CharEncoding transformer - two transformers and so on. most of the functionality in Input and Ouput variants will be repeated (because this functionality don't depend on input/output mode) and in addition to the current large lists of passed calls like the:
vIsEOF (WithEncoding h _) = vIsEOF h vMkIOError (WithEncoding h _) = vMkIOError h vReady (WithEncoding h _) = vReady h vIsReadable (WithEncoding h _) = vIsReadable h
we will get the same lists in 2 or 3 repetitions!!!
the common operations should be members of a separate superclass. I have in mind a structure like this: class Stream h where streamEOF :: h -> IO Bool streamReady :: h -> IO Bool streamClose :: h -> IO () class InputByteStream h where streamGet :: h -> IO Word8 ... class InputBlockStream h where streamGetBuf :: h -> Int -> Ptr Word8 -> IO Int ... class InputMemoryStream h where streamGetMem :: h -> IO (Ptr Word8) ... there's no duplication, just more structure. By the way, I like your idea of exposing the difference between MemoryStream and ByteStream. I'm not so sure about the difference between BlockStream and ByteStream - I think BlockStream should be the lowest level, and all the ByteStream operations can be provided on BlockStreams (or MemoryStremas) by reading/writing one byte at a time.
3) i don't think that we can completely throw away the r/w streams, they can be required for example for database-style access.
This doesn't stop you from having read/write streams, but it means that you could implement read-only and write-only buffering without the complication that comes with the possibility of read/write. You have to implement read/write buffering separately from read-only and write-only buffering. For example, you could have an InOutBufferedStream transformer that layers on top of two underlying buffered streams, and remembers which one was used last. If an operation occurs on the other one, then the in-use buffer is flushed first. This means you only pay the penalty of checking & flushing when you need to use this read/write transformer, rather than in every buffered stream.
why ByteStream implements both the byte and text i/o? i think that in most cases people are still using latin-1 text i/o - i.e. each char is just 8 bits without any encoding. because that type of text i/o don't need any complex implememtation, each time when byte i/o is implemented, text i/o springs automatically. on the other side, utf-8 encoding is rare and therefore separate transformer is used to implement it - it transforms each char i/o call into several byte i/o calls.
i definitely against implementing text i/o only through the encoding transformer because it will slowdown the i/o while in 90% cases encoding will not be used.
I don't think this is the way to go. We should assume that text encoding/decoding is the norm, rather than optimising (heavily) for Latin-1. By all means have a specialised Latin-1 stream transformer, which can be very efficient, but don't build it into the byte stream class. Byte streams should deal in bytes, not Chars. UTF-8 might be rare at the moment, but it will be the norm soon. The library should provide fast text encoding/decoding, which means doing it via buffers, and probably using iconv, since we don't want to re-implement the encodings ourselves - for example what happens with UTF-8 encoding errors in your implementation? This gives us a dilemna - since encoding and decoding needs to operate directly on buffers, it needs direct access to the buffer. So it looks like encoding/decoding and buffering must be combined (this is what I did in my library). But, you don't want to add this buffering to a memory stream, so perhaps buffered streams should be instances of MemoryStreams, and text encoding/decoding should layer on MemoryStreams only? (is this what you did?)
SM> This will improve SM> performance too - your Stream class has dictionaries with 20+ elements.
here you are king - i don't know whether it's better to have one class with 20 methods or 2 classes with 5 methods each in the function context?
see above - I'd like to see some measurements here.
SM> I see that buffering works on vPutChar/vGetChar, and yet you seem to be SM> buffering bytes - which is it? Are you supposed to buffer before or SM> after doing character encoding? It seems before, because otherwise SM> buffering will strip out all but the low 8 bits of each character. SM> Using a more explicit type structure would help a lot here.
buffer contains bytes, which are read/written by the getbyte/putbyte operations as well as the all text i/o. this is latin1-only solution, of course. if one need utf-8 encoding, he need to apply CharEncoding transformer
see above - I think this is wrong. Cheers, Simon