On 23/11/06, Jason Dagit wrote:

A comment on that video said:

----- BEGIN QUOTE ---- It seems to me that STM creates new problems with composability. You create two classes of code: atomic methods and non atomic methods.

Nonatomic methods can easily call atomic ones – the compiler could even automatically inject the atomic block if the programmer forgot.

Atomic methods and blocks cannot be allowed to call nonatomic code. The nonatomic code could do I/O or other irrevocable things that would be duplicated when the block had to retry. ---- END QUOTE ----

I imagine an example like this (some pseudo code for a side effect happy OO language):

class Foo { protected int counter; // assume this gets initialized to 0 public doSomething() { atomic{ counter++; Console.Write("called doSomething execution# " + counter); // something which could cause the transaction to restart } } public doOtherThing() { atomic{ doSomething(); // something which could cause the transaction to restart } } }

Now imagine doSomething gets restarted, then we see the console output once each time and counter gets incremented. So one solution would be to move the side effects (counter++ and the console write) to happen before the atomic block. This works for doSomething, but now what if we called doOtherThing instead? We're back to having the extra side-effects from the failed attempts at doSomething, right? We just lost composability of doSomething? I'm assuming counter is only meant to be incremented once per successful run of doSomething and we only want to see the output to the log file once per successful run, but it needs to come before the log output inside doSomething so that the log makes sense.

I realize STM is not a silver bullet, but it does seem like side-effects do not play nicely with STM. What is the proposed solution to this? Am I just missing something simple? Is the solution to make it so that Console.Write can be rolled back too?

Thanks, Jason

The solution is to simply not allow side effecting computations in transactions. They talk a little about it in the video, but perhaps that's not clear. The only side effects an atomic STM transaction may have are changes to shared memory. Another example in pseudocode: atomic x <- launchMissiles if (x < 5) then retry This is obviously catastrophic. If launchMissiles has the side effect of launching a salvo of missiles, and then the retry occurs, it's unlikely that rolling back the transaction is going to be able to put them back on the launchpad. Worse yet, if some variable read in launchMissiles changes, the transaction would retry, possibly causing a second salvo of missiles to be launched. So you simply disallow this. The content of a transaction may only include reads and writes to shared memory, along with pure computations. This is especially easy in Haskell, because one simply uses a new monad STM, with no way to lift IO actions into that monad, but atomically :: (STM a -> IO a) goes in the other direction, turning a transaction into IO. In other languages, you'd want to add some static typechecking to ensure that this constraint was enforced. - Cale

(Dropping Haskell@hakell.org) Hi, We've not yet looked at I/O in detail in Haskell, but there's a paper from a few years back where I experimented with ways of integrating I/O with an earlier implementation of atomic blocks in Java. http://research.microsoft.com/~tharris/papers/2005-scp.pdf The basic idea is to provide a way for a transaction to call into transaction-aware libraries. The libraries can register callbacks for if the transaction commits (to actually do any "O") and for if the transaction aborts (to re-buffer any "I" that the transaction has consumed). In addition, a library providing access to another transactional abstraction (e.g. a database supporting transactions) can perform a 2-phase commit that means that the memory transaction and database transaction either both commit or both abort. Of course, these solutions don't deal with the question of atomic blocks that want to perform output (e.g. to the console) and receive input in response to that. My view at the moment is _that does not make sense in an atomic block_ -- the output and input can't be performed atomically because the intervening state must be visible for the user to respond to. We also briefly experimented with extending the SXM system Maurice Herlihy worked on at MSR Cambridge to support transactions that include accesses to the file system and registry -- http://msdn2.microsoft.com/en-us/library/aa366295.aspx describes the TxF and TxR systems it was built over. Some other interesting work in this area is Elliot Moss' papers on "open nested" transactions -- these provide another building block at the same level as the Java system I mentioned: library writers can use them with care to extend the range of things that can be done inside an atomic block. Cheers, Tim -----Original Message----- From: haskell-bounces@haskell.org [mailto:haskell-bounces@haskell.org] On Behalf Of Benjamin Franksen Sent: 24 November 2006 03:16 To: haskell@haskell.org Cc: haskell-cafe@haskell.org Subject: [Haskell] Re: [Haskell-cafe] SimonPJ and Tim Harris explain STM - video [sorry for quoting so much, kinda hard to decide here where to snip] Cale Gibbard wrote:

On 23/11/06, Jason Dagit wrote:

...

A comment on that video said:

----- BEGIN QUOTE ---- It seems to me that STM creates new problems with composability. You create two classes of code: atomic methods and non atomic methods.

Nonatomic methods can easily call atomic ones ? the compiler could even automatically inject the atomic block if the programmer forgot.

Atomic methods and blocks cannot be allowed to call nonatomic code. The nonatomic code could do I/O or other irrevocable things that would be duplicated when the block had to retry. ---- END QUOTE ----

I imagine an example like this (some pseudo code for a side effect happy OO language):

class Foo { protected int counter; // assume this gets initialized to 0 public doSomething() { atomic{ counter++; Console.Write("called doSomething execution# " + counter); // something which could cause the transaction to restart } } public doOtherThing() { atomic{ doSomething(); // something which could cause the transaction to restart } } }

Now imagine doSomething gets restarted, then we see the console output once each time and counter gets incremented. So one solution would be to move the side effects (counter++ and the console write) to happen before the atomic block. This works for doSomething, but now what if we called doOtherThing instead? We're back to having the extra side-effects from the failed attempts at doSomething, right? We just lost composability of doSomething? I'm assuming counter is only meant to be incremented once per successful run of doSomething and we only want to see the output to the log file once per successful run, but it needs to come before the log output inside doSomething so that the log makes sense.

I realize STM is not a silver bullet, but it does seem like side-effects do not play nicely with STM. What is the proposed solution to this? Am I just missing something simple? Is the solution to make it so that Console.Write can be rolled back too?

The solution is to simply not allow side effecting computations in transactions. They talk a little about it in the video, but perhaps that's not clear. The only side effects an atomic STM transaction may have are changes to shared memory.

Another example in pseudocode:

atomic x <- launchMissiles if (x < 5) then retry

This is obviously catastrophic. If launchMissiles has the side effect of launching a salvo of missiles, and then the retry occurs, it's unlikely that rolling back the transaction is going to be able to put them back on the launchpad. Worse yet, if some variable read in launchMissiles changes, the transaction would retry, possibly causing a second salvo of missiles to be launched.

So you simply disallow this. The content of a transaction may only include reads and writes to shared memory, along with pure computations. This is especially easy in Haskell, because one simply uses a new monad STM, with no way to lift IO actions into that monad, but atomically :: (STM a -> IO a) goes in the other direction, turning a transaction into IO. In other languages, you'd want to add some static typechecking to ensure that this constraint was enforced.

This is of course the technically correct answer. However, I suspect that it may not be completely satisfying to the practitioner. What if you want or even need your output to be atomically tied to a pure software transaction? One answer is in fact "to make it so that Console.Write can be rolled back too". To achieve this one can factor the actual output to another task and inside the transaction merely send the message to a transactional channel (TChan): atomic $ do increment counter counterval <- readvar counter sendMsg msgChan ("called doSomething execution# " ++ show counterval) -- something which could cause the transaction to restart Another task regularly takes messages from the channel and actually outputs them. Of course the output will be somewhat delayed, but the order of messages will be preserved between tasks sending to the same channel. And the message will only be sent if and only if the transaction commits. Unfortunately I can't see how to generalize this to input as well... Cheers Ben _______________________________________________ Haskell mailing list Haskell@haskell.org http://www.haskell.org/mailman/listinfo/haskell

On Fri, Nov 24, 2006 at 08:22:36AM +0000, Simon Peyton-Jones wrote:

I have also toyed with adding

retryWith :: IO a -> STM ()

The idea here is that the transction is undone (i.e. just like the 'retry' combinator), then the specified action is performed, and then the transaction is retried. Again no atomicity guarantee. If there's an orElse involved, both actions would get done.

Unlike onCommit, onRetry adds new power. Suppose you have a memory buffer, with an STM interface: getLine :: Buffer -> STM STring

[Sorry for a long email, I had no time to make it short ;-)] Another example would be my experiments with supporting time in STM, ie. functions: retryIfBefore :: UTCTime -> STM () retryIfAfter :: UTCTime -> STM () (Current code can be obtained with "darcs get" from http://www.uncurry.com/haskell/stm-time/. BTW, shout if you want such a library - it will motivate me to finally release it officially). The naive implementation could hold UTCTime in a single TVar and update this variable in a loop, say, 100 times a second. Of course, with many threads using retryIfBefore/retryIfAfter this would cause too many retries. In my implementation I split the time TVar into a list of TVars, each holding a bit of time representation. The retryIf* functions are written in such a way, that they retry as soon as they can tell that it's too early or too late. This way the number of retries for (retryIfBefore then) is at most the length of suffix of bits that differ in representation of "now" and "then". But there is still a problem with accuracy. Ideally, we would like to be as accurate as possible. One solution goes like this: if retryIfBefore retries because the time value stored in variables is too low, let's allow it to notify the manager thread what UTCTime it is waiting for, so it can schedule to update the variables exactly at this moment. That's where retryWith would help. Right now I am using something named autonomous transactions: autonomously :: Bool -> STM a -> STM () This basically forks a new thread to perform the given transaction outside of the current transaction. To be fair, I am not sure it is sound - as you can imagine, the implementation uses some dirty tricks like unsafeIOToSTM. I haven't checked what would happen if I used some variables created in the surrounding transaction. BTW, implementing STM.Time was very instructive for me. It made me realize that I didn't understand STM as well as I thought. Perhaps it could be made in a nice tutorial, if it wasn't so riddled with unsafish stuff: unsafeIOToSTM mentioned above, and unsafePerformIO used to initialize a top-level variable *and* spawn a manager thread. Here retryWith could also help - I fork the manager thread with it.

I have not implemented either of these, but I think they'd be cool.

I agree especially about retryWith. But I think it's name should include a "danger! sign", because when used wrong, it can "break" the nice properties of STM and cause very surprising bugs. For me one good "danger" indicator is "IO", so perhaps "retryWithIO" ? Best regards Tomasz

I was inspired by Simon's post to kludge up a working prototype that does what is discussed: Simon Peyton-Jones wrote:

| The basic idea is to provide a way for a transaction to call into transaction-aware libraries. The libraries | can register callbacks for if the transaction commits (to actually do any "O") and for if the transaction | aborts (to re-buffer any "I" that the transaction has consumed). In addition, a library providing access | to another transactional abstraction (e.g. a database supporting transactions) can perform a 2-phase | commit that means that the memory transaction and database transaction either both commit or both | abort.

Yes, I have toyed with extending GHC's implementation of STM to support

onCommit :: IO a -> STM ()

The idea is that onCommit would queue up an IO action to be performed when the transaction commits, but without any atomicity guarantee. If the transaction retries, the action is discarded. Now you could say

I have also toyed with adding

retryWith :: IO a -> STM ()

The idea here is that the transction is undone (i.e. just like the 'retry' combinator), then the specified action is performed, and then the transaction is retried. Again no atomicity guarantee. If there's an orElse involved, both actions would get done.

It would also make it possible to count how many retries happened: atomic (<transaction> `orElse` retryWith <increment retry counter>)

I have not implemented either of these, but I think they'd be cool.

Simon

The prototype is: {- November 24th, 2006 Demonstration Code by Chris Kuklewicz Usual 3 clause BSD Licence Copyright 2006 This is inspired by a post by Simon Peyton-Jones on the haskell-cafe mailing list, in which the type and semantics of onCommit and withRetry were put forth. The semantics of printing the contents of the TVar "v" created in test via retryWith may or may not be well defined. With GHC 6.6 I get *AdvSTM> main "hello world" "retryWith Start" ("retryWith v",7) "Flipped choice to True to avoid infinite loop" "onCommit Start" ("onCommit v",42) ("result","foo") "bye world" Aside from that I think the unsafeIOToSTM is not really unsafe here since it writes to privately created and maintained variables. Since the implementation is hidden it could be changed from ReaderT to some other scheme. Once could also use MonadBase from http://haskell.org/haskellwiki/New_monads/MonadBase to help with the lifting, but this has been commented out below. TODO: figure out semantics of catchAdv. At least it compiles... -} module AdvSTM(MonadAdvSTM(..),AdvSTM,retryWith) where -- import MonadBase import Control.Exception(Exception) import Control.Monad(MonadPlus(..),liftM) import Control.Monad.Reader(MonadReader(..),ReaderT,runReaderT,lift,asks) import Control.Concurrent.STM(STM,orElse,retry,catchSTM,atomically) import Control.Concurrent.STM.TVar(TVar,newTVarIO,newTVar,readTVar,writeTVar) import GHC.Conc(unsafeIOToSTM) import Data.IORef(IORef,newIORef,readIORef,writeIORef,modifyIORef) import Data.Typeable(Typeable) import Data.Generics(Data) class MonadAdvSTM m where onCommit :: IO a -> m () onRetry :: IO a -> m () orElseAdv :: m a -> m a -> m a retryAdv :: m a atomicAdv :: m a -> IO a catchAdv :: m a -> (Exception -> m a) -> m a liftAdv :: STM a -> m a -- Export type but not constructor! newtype AdvSTM a = AdvSTM (ReaderT (CommitVar,RetryVar) STM a) deriving (Functor,Monad,MonadPlus,Typeable) type CommitVar = TVar ([IO ()]->[IO ()]) type RetryVar = IORef ([IO ()]->[IO ()]) {- Since lifting retry and `orElse` gives the semantics Simon wants, use deriving MonadPlus instead instance MonadPlus AdvSTM where mzero = retryAdv mplus = orElseAdv -} -- instance MonadBase STM AdvSTM where liftBase = AdvSTM . lift retryWith :: IO a -> AdvSTM b retryWith io = onRetry io >> retryAdv instance MonadAdvSTM AdvSTM where onCommit io = do cv <- AdvSTM $ asks fst old <- liftAdv $ readTVar cv liftAdv $ writeTVar cv (old . ((io >> return ()):)) onRetry io = do rv <- AdvSTM $ asks snd liftAdv $ unsafeIOToSTM $ modifyIORef rv (\ old -> old . ((io >> return ()):) ) {- orElseAdv (AdvSTM a) (AdvSTM b) = {- If a retries then its onRetry commands are kept on the list of actions to do if the whole command fails. It would be possible to save the "rv" and use unsafeIOToSTM to implement a different policy here -} AdvSTM $ do env <- ask lift $ (runReaderT a env) `orElse` (runReaderT b env) -} orElseAdv = mplus retryAdv = liftAdv retry -- the same as retryAdv = mzero atomicAdv = runAdvSTM catchAdv (AdvSTM action) handler = let h env error = let (AdvSTM cleanup) = handler error in runReaderT cleanup env in AdvSTM $ do env <- ask lift $ catchSTM (runReaderT action env) (h env) liftAdv = AdvSTM . lift -- This replaces "atomically" runAdvSTM :: AdvSTM a -> IO a runAdvSTM (AdvSTM action) = do cv <- newTVarIO id rv <- newIORef id let wrappedAction = (runReaderT (liftM Just action) (cv,rv)) `orElse` (return Nothing) loop = do result <- atomically $ wrappedAction case result of Just answer -> do cFun <- atomically (readTVar cv) sequence_ (cFun []) return answer Nothing -> do rFun <- readIORef rv writeIORef rv id -- must reset the list sequence_ (rFun []) loop loop -- Example code using the above: test :: TVar Bool -> AdvSTM String test todo = do onCommit (print "onCommit Start") onRetry (print "onRetry Start") v <- liftAdv $ newTVar 7 liftAdv $ writeTVar v 42 onCommit (atomically (readTVar v) >>= \x->print ("onCommit v",x)) onRetry (atomically (readTVar v) >>= \x->print ("onRetry v",x)) choice <- liftAdv $ readTVar todo case choice of True -> return "foo" False -> retryWith $ do atomically (writeTVar todo True) print "Flipped choice to True to avoid infinite loop" -- Example similar to Simon's suggested example: countRetries :: IORef Int -> AdvSTM a -> AdvSTM a countRetries ioref action = let incr = do old <- readIORef ioref writeIORef ioref $! (succ old) in action `orElseAdv` (retryWith incr) -- Load this file in GHCI and execute main to run the test: main = do print "hello world" todo <- newTVarIO False counter <- newIORef 0 result <- runAdvSTM (test todo) print ("result",result) print "bye world"

I have also toyed with adding

retryWith :: IO a -> STM ()

The idea here is that the transction is undone (i.e. just like the 'retry' combinator), then the specified action is performed, and then the transaction is retried. Again no atomicity guarantee. If there's an orElse involved, both actions would get done.

Unlike onCommit, onRetry adds new power. Suppose you have a memory buffer, with an STM interface: getLine :: Buffer -> STM STring

This is the way to do transactional input: if there is not enough input, the

Simon Peyton-Jones wrote: transaction retries; and the effects of getLine aren't visible until the transaction commits. The problem is that if there is not enough data in the buffer, getLine will retry; but alas there is no way at present to "tell" someone to fill the buffer with more data.

onRetry would fix that. getLine could say if <not enough data> then retryWith <fill-buffer action>

It would also make it possible to count how many retries happened: atomic (<transaction> `orElse` retryWith <increment retry counter>)

I have not implemented either of these, but I think they'd be cool.

Simon

After seeing how close I could come to creating onRetry/retryWith I have a question about the semantics of your idea for retryWith. Normally after a retry the STM block is rolled back and put to sleep and will only be awakened and re-executed if one of the STM variables it had read from is committed to by a different STM block. What about retryWith ? Will the STM block be put to sleep under the same conditions? Can the IO action given to retryWith include commits to STM variables?

Hi,

After seeing how close I could come to creating onRetry/retryWith I have a question about the semantics of your idea for retryWith.

Normally after a retry the STM block is rolled back and put to sleep and will only be awakened and re-executed if one of the STM variables it had read from is committed to by a different STM block.

What about retryWith ? Will the STM block be put to sleep under the same conditions? Can the IO action given to retryWith include commits to STM variables?

Starting with this last question -- yes, the example of an STM GetLine using retryWith to pull more input into a buffer relies on an atomic block in the IO action to update the buffer. There are some interesting questions to think about the semantics of "retryWith". The semantics of "retry" in the PPoPP paper say nothing about putting threads to sleep -- it would be correct for an implementation to spin repeatedly executing an "atomic" block until it completes without calling "retry". What should happen with "retryWith" -- should we introduce blocking & wake-up into the semantics, or say that the "retryWith" actions are collected together, executed in place of the transaction (if it does ultimately retry) and then the transaction re-attempted? For simplicity (and to leave flexibility to the implementation) I'd prefer to keep blocking & wake-up out of the semantics. Taking that option, suppose we have "atomic { X }" and X retries with IO action Y. I think I'm saying that that would be equivalent to "Y ; atomic { X }". What are the consequences of this? In the GetLine example it means that an implementation that does put threads to sleep must detect conflicts between the first execution of X and any transactions performed in Y. Are there any interesting problems that come up if "Y" performs transactions that use "retry" or "retryWith"? Tim

| In many useful cases, such as the getLine example, the Y action will have its | own atomic {} block. In which case the semantics of when it is allowed to | re-attempt X are what is important. If you require (Y) to complete before | re-attempting (X) then you get an infinite regression where every (atomic block) | fails with (retryWith (next atomic block)), and nothing is ever re-attempted. | This is why "retryWith Y" meaning rollback X and do "Y >> atomic X" is the wrong | implementation. I don't agree. I think it's quite reasonable. Not many atomic blocks will finish with retryWith. Of course there is a possibility of an infinite loop, but we already have that: f x = f x. Of course, Y can always choose to do a forkIO, but it shouldn't hav to. For me the only difficulty is the implementation. We'd like to block X on the TVars it read (as usual), *unless* executing Y wrote to any of them. That requires a bit more cleverness in the commit code, but not a great deal I think. Simon

this thread reminds me about something that I wanted to ask you. if I recall correctly, most of the literature references in STM papers are recent, so I wondered whether you are aware of this one: NAMING AND SYNCHRONIZATION IN A DECENTRALIZED COMPUTER SYSTEM SourceTechnical Report: TR-205 Year of Publication: 1978 Author D. P. Reed I'm not entirely sure where I got my version from (it was mentioned as a cornerstone in Alan Kay et al s latest project, Croquet, on which Reed is a collaborator: http://www.opencroquet.org/ ), but here is the abstract: http://portal.acm.org/citation.cfm?coll=GUIDE&dl=GUIDE&id=889815 (note that it mentions both grouping of updates, and synchronized composition of modules with local synchronization constraints) and this might be the official site for the scanned copy (?): http://www.lcs.mit.edu/publications/specpub.php?id=773 just wondering, claus

also on the multi-version approach. but the particular reason I mentioned this was that it also tried to address the issue of composing transactions (discussion of which is explicitly excluded from the section 1.2 you refer to). for instance, compare the discussion and examples in "Composable Memory Transactions" with the one in section 3.11 (pp 80-) of Reed's thesis. not that STM isn't nicer, more complete, and all that;-) but Reed's ideas seemed rather more advanced than I had expected from the discussion of related work in the STM paper. Claus

Interesting reference. I had never heard of it. From reading section 1.2 it sounds like an early description of > the optimistic approach to implementing atomic transactions (which is itself a well-studied field).

Simon

| NAMING AND SYNCHRONIZATION IN A | DECENTRALIZED COMPUTER SYSTEM | | SourceTechnical Report: TR-205 | Year of Publication: 1978 | Author D. P. Reed | | I'm not entirely sure where I got my version from (it was mentioned | as a cornerstone in Alan Kay et al s latest project, Croquet, on which | Reed is a collaborator: http://www.opencroquet.org/ ), but here is | the abstract: | | http://portal.acm.org/citation.cfm?coll=GUIDE&dl=GUIDE&id=889815 | | (note that it mentions both grouping of updates, and synchronized | composition of modules with local synchronization constraints) | | and this might be the official site for the scanned copy (?): | | http://www.lcs.mit.edu/publications/specpub.php?id=773

Hi Liyang HU you wrote:

On 23/11/06, Benjamin Franksen wrote:

...

One answer is in fact "to make it so that Console.Write can be rolled back too". To achieve this one can factor the actual output to another task and inside the transaction merely send the message to a transactional channel (TChan):

So, you could simply return the console output as (part of) the result of the atomic action. Wrap it in a WriterT monad transformer, even.

But this would break atomicity, wouldn't it? Another call to doSomething from another task could interrupt before I get the chance to do the actual output. With a channel whatever writes will happen in the same order in which the STM actions commit (which coincides with the order in which the counters get incremented).

...

Another task regularly takes messages from the channel

With STM, the outputter task won't see any messages from the channel until your main atomic block completes, after which you're living in IO-land, so you might as well do the output yourself.

Yeah, right. Separate task might still be preferable, otherwise you have to take care not to forget to actually do the IO after each transaction. I guess it even makes sense to hide the channel stuff behind some nice abstraction, so inside the transaction it looks similar to a plain IO action: output port msg The result is in fact mostly indistiguishable from a direct IO call due to the fact that IO is buffered in the OS anyway.

Pugs/Perl 6 takes the approach that any IO inside an atomic block raises an exception.

...

Unfortunately I can't see how to generalize this to input as well...

The dual of how you described the output situation: read a block of input before the transaction starts, and consume this during the transaction. I guess you're not seeing how this generalises because potentially you won't know how much of the input you will need to read beforehand... (so read all available input?(!) You have the dual situation in the output case, in that you can't be sure how much output it may generate / you will need to buffer.)

You say it. I guess the main difference is that I have a pretty good idea how much data is going to be produced by my own code, and if it's a bit more than I calculated then the whole process merely uses up some more memory, which is usually not a big problem. However, with input things are different: in many cases the input length is not under my control and could be arbitrarily large. If I read until my buffer is full and I still haven't got enough data, my transaction will be stuck with no way to demand more input. (however, see below)

input <- hGetContent file atomic $ flip runReaderT input $ do input <- ask -- do something with input return 42

(This is actually a bad example, since hGetContents reads the file lazily with interleaved IO...)

In fact reading everything lazily seems to be the only way out, if you don't want to have arbitrary limits for chunks of input. OTOH, maybe limiting the input chunks to some maximum length is a good idea regardless of STM and whatnot. Some evil data source may want to crash my process by making it eat more and more memory... So, after all you are probably right and there is an obvious generalization to input. Cool. Cheers Ben

Liyang HU wrote:

On 24/11/06, Benjamin Franksen wrote:

...

I have a pretty good idea how much data is going to be produced by my own code, and if it's a bit more than I calculated then the whole process merely uses up some more memory, which is usually not a big problem. However, with input things are different:

Really? I'd have said that I have a pretty good idea how much data is going to be consumed by my own code, and if it's a bit more than I calculated then I'd merely read some more at the beginning (putting any unused bits back on the input queue afterwards of course), which is usually not a big problem. :)

Yes, I do get your point. It's easier to allocate a larger buffer for your output as needed, than to anticipate how much input you might require. I'd still claim they're different instances of the same scheme though!

...

[If] I still haven't got enough data, my transaction will be stuck with [no way to demand more input.

Take your output channel idea, and use that for input too? (Separate thread to read the input and place it at the end of some queue.) You would basically retry and block (or rather, STM would do the latter for you) if you haven't enough data, until more came along.

Right. I couldn't see it at first but the I and O really are dual to each other. Thanks for pointing this out -- it seems STM is even more useful than I thought. Cheers Ben

On Fri, Nov 24, 2006 at 10:31:27AM +0100, Lemmih wrote:

Worked for me with mplayer+w32codecs.

Oops! I missed the "Download" link and tried to download through "Watch" ;-) Thanks! Best regards Tomasz