On Wed, Mar 31, 2010 at 2:12 PM, John Lato
Hi Dave,
From: David Leimbach
I'm looking at iteratee as a way to replace my erroneous and really inefficient lazy-IO-based backend for an expect like Monad DSL I've been working for about 6 months or so now on and off.
The problem is I want something like:
expect "some String" send "some response"
to block or perhaps timeout, depending on the environment, looking for "some String" on an input Handle, and it appears that iteratee works in a very fixed block size. While a fixed block size is ok, if I can put back unused bytes into the enumerator somehow (I may need to put a LOT back in some cases, but in the common case I will not need to put any back as most expect-like scripts typically catch the last few bytes of data sent before the peer is blocked waiting for a response...)
I'm quite sure I don't know what you're trying to do. The only time I can think of needing this is if you're running an iteratee on a file handle, keeping the handle open, then running another iteratee on it. Is this what you're doing? If so, I would make a new run function:
runResidue :: (Monad m, SC.StreamChunk s el) => IterateeG s el m a -> m (a, s) runResidue iter = runIter iter (EOF Nothing) >>= \res -> case res of Done x s -> return (x, s) Cont _ e -> error $ "control message: " ++ show e
This function will return the unused portion of the stream, then you can do this:
enumResidue :: Handle -> s -> EnumeratorGM s el m a enumResidue h s = enumPure1Chunk s >. enumHandle h
Is this what you need? If I'm completely wrong about what you're trying to do (or you're using multiple threads), there are other options.
The problem is I am not sure what it was I needed to get started to begin with. For a moment it seemed that I could be throwing out data that's been read, but not yet fed to an iteratee step. If that's not the case, I'll never need to put back any characters. The attoparsec-iteratee example posted, plus some experimentation has led me to believe I don't need to worry about this sort of thing for the kind of processing I'm looking to do.
You also may want to look at iteratee-HEAD. The implementation has been cleaned up a lot, the block sizes are user-specified, and there's an exception-based, user-extensible mechanism for iteratees to alter enumerator behavior.
That's very compelling. Here's the properties of the system I'm trying to build (in fact I've built this system with Haskell already months ago, but trying to evaluate if iteratee can fix problems I've got now) 1. Must have an expect-like language to a subprocess over a pair of Handles such that I can query what is normally a command line interface as a polling refresher thread to a cache. Note that there may be many sub-processes with a poller/cache (up to 6 so far). Data produced is dumped as records to stdout from each thread such that the process that spawned this haskell program can parse those records and update it's view of this particular part of the world. 2. Must be able to interleave commands between polled record data from the processes in 1. These commands come in over this process's stdin from the program that started the Haskell program. 3. The polling process in 1, must be able to respond to a timeout situation. In this system, a cable could become disconnected or a part of the system could become unavailable, or a part of the system underneath could become unreliable and require a restart to guarantee the serviceability of the whole system. I have 1 and 2 working fine in a current iteration of the system, but because we're dealing with a pretty complex stack, 3 is really necessary too, and I've conquered my timeout issues from earlier in a reasonable enough way. The problem is that the timeout "handler" apparently runs into problems that I think stem from lazyIO not having been evaluated on a handle yet, but the timeout handler has invalidated that handle, and then that thunk, which was partially applied, is now talking to some broken value. As far as I know, I can't go back and prevent previously bound thunks that have the wrong handle from executing, if that is truly what's happening. What I'd like to do is prevent that situation from ever happening to begin with, or at least rule it out. I'm hoping version 2 of this system to be based on iteratee and avoid this sort of problem at all. The alternative is to write this in another language, but that throws out a lot of nice and simple code that's present in the non IO bits of this code, for dealing with parsing and data serialization. Dave
Sincerely, John