Re: [Haskell-cafe] External system connections

Alright, I'll have to think on this some more but I think we're speaking the same language now - and what's more I even understand it! Thanks again for all your help, Rich On Sun, Jul 10, 2011 at 6:46 PM, Brandon Allbery wrote:

On Sun, Jul 10, 2011 at 20:19, Richard Wallace wrote:

...

On Sun, Jul 10, 2011 at 4:38 PM, Brandon Allbery wrote: Ok, I can see that.  Though I was thinking that the worker threads would send the request back to the dispatcher by way of the same Chan the dispatcher reads requests from.  Obviously I was thinking the dispatcher would use the original token to filter requests in the Chan.  If I understand what you are talking about, the dispatcher would do the token renewal in a separate thread, continuing to process

The same Chan is used to send a request; the thread processing token renewal might or might not be otherwise a normal worker thread, but it's separated out as a Maybe ThreadId instead of being in a pool, because (a) there can only be zero or one of them, and (b) if it's not Nothing then the dispatcher thread accepts only token renewals.  (This actually requires either multiple Chans or something more complex than a normal Chan, since you can't filter a Chan based on the type of message.)  You also need some way to block the sender, which suggests that a message written down a Chan must include an MVar which will be signaled when the operation is complete.  This suggests to me something along the lines of

...

data WorkRequest = SOAPData ... (MVar Bool)                  | TokenRequest Token (MVar Token) --               | ReadyForWork (MVar WorkRequest)

where the requestor allocates an MVar, writes it as part of the WorkRequest, and then does a takeMVar to wait for the response.  The dispatcher reads WorkRequests, dispatches any it can to available workers, and queues the rest internally; if it's a TokenRequest then it's queued separately and all SOAPData requests get queued regardless of whether there are free workers.  When the single token processor returns, all entries in the TokenRequest queue get awakened (putMVar threadMVar newToken) and normal processing of the standard request queue resumes.

Or you can see if the pool hackage handles the ugly details here automatically; I haven't looked.

...

If the pool is of threads, how do you start the threads?  How do you submit work to the threads?  The only way I know of in Haskell of creating threads to do work is forkIO.  That takes a function and runs to completion.  Would a worker thread just be one that loops forever

Yes; the dispatcher keeps a list of workers, which are forkIO-d threads that are waiting on an MVar or Chan for work to do.  When they receive something, they go off and do it, write the result into another MVar or Chan which was specified in the request, and go back to waiting on the initial MVar/Chan for something to do.  If the list is shorter than the maximum, more workers are forkIO-d to fill it as needed; if longer, idle workers are sent "shut down" requests.  (The latter is "polite" handling of program shutdown, and also allows for the pool size to be modified dynamically if needed.)  I think doing this right also requires that a worker that's ready for more work explicitly check in, so the dispatcher knows it's available; that could be handled by an additional WorkRequest type (see commented-out line above, where a worker that's ready to handle another request passes its input MVar to the dispatcher)... but there may be better ways; I have some grasp of concurrency, but my Haskell library fu is still somewhat weak.  Hopefully someone else will jump in if appropriate.

(You can see how quickly this becomes complex, though; if the canned solution does what you need, you might want to avoid reinventing this particular wheel unless you're doing it for educational purposes.)

-- brandon s allbery                                      allbery.b@gmail.com wandering unix systems administrator (available)     (412) 475-9364 vm/sms