External system connections

Hello all, I'm trying to understand how to properly structure a connection to an external system. I'm writing an application that processes requests (it's an IRC bot - ya, I've looked at lambabot but it's a bit beyond my current understanding and I'm really trying to learn this stuff and find the best way to do that is to write it myself) and sends requests to an external system, via SOAP. The SOAP requests should be sent on separate threads. The SOAP server requires an initial request for authentication be made to obtain a token used on subsequent requests. There are three ways I can structure this. One is to do the authentication with the SOAP server when the application starts up and just use that token for all subsequent requests. This will fall over if the token times out and I'd have to restart the application. Another way is to do authenticate with the server and get a new token for each request. This is obviously really inefficient. What I'd like to do is something a bit smarter. When a request to the SOAP server is to be made, if we have a token then we try and use it. If it fails, we reauthenticate and get a new token. When establishing a new token, other threads trying to send SOAP requests should block until a new token is available. There are other conditions that should be handled - such as an exponential back-off when the SOAP server can't be reached - but I feel like reestablishing the authentication token is really the key concern here. Unfortunately, I have no idea right now how to write this in Haskell. Any pointers would be awesome. Thanks! Rich

You don't need to do it, it is already done =). See the pool package by Michael Snoyman on Hackage [1]. More specifically, see createPoolCheckAlive [2]. Cheers, [1] http://hackage.haskell.org/package/pool [2] http://hackage.haskell.org/packages/archive/pool/0.1.0.2/doc/html/Data-Pool.... -- Felipe.

On 07/10/11 15:37, Felipe Almeida Lessa wrote:
You don't need to do it, it is already done =). See the pool package by Michael Snoyman on Hackage [1]. More specifically, see createPoolCheckAlive [2].
Cheers,
[1] http://hackage.haskell.org/package/pool [2] http://hackage.haskell.org/packages/archive/pool/0.1.0.2/doc/html/Data-Pool....
How do people use this stuff? The README is empty, there's no documentation, and none of the functions are commented. I'm forced to conclude that there are people capable of looking at this, createPoolCheckAlive :: MonadControlIO m => IO a -> (a -> IO ()) -> Int -> (Pool a -> m b) -> (a -> IO Bool) -> m b and knowing immediately what it does. I'm still a beginner, and I don't think I have a use for this package (what does it do?), but I've run into similar situations with other packages and would genuinely like to know (as opposed to just complain about it).

On Mon, Jul 11, 2011 at 11:27, Michael Orlitzky
How do people use this stuff? The README is empty, there's no documentation, and none of the functions are commented. I'm forced to conclude that there are people capable of looking at this,
I can *almost* work it out just from the type and knowing what it should be doing. But in this case, the main problem is that it was split off of yesod (http://www.yesodweb.com/), and it was only documented in the context of that. -- brandon s allbery allbery.b@gmail.com wandering unix systems administrator (available) (412) 475-9364 vm/sms

On Mon, Jul 11, 2011 at 12:27 PM, Michael Orlitzky
How do people use this stuff? The README is empty, there's no documentation, and none of the functions are commented. I'm forced to conclude that there are people capable of looking at this,
createPoolCheckAlive :: MonadControlIO m => IO a -> (a -> IO ()) -> Int -> (Pool a -> m b) -> (a -> IO Bool) -> m b
and knowing immediately what it does.
Yes, there's a documentation problem. But you can guess by the types =). IO a: opens a new resource a. a -> IO (): disposes the given resource. Int: maximum number of open resources. Pool a -> m b: the action you want to execute; after it executes, the pool is destroyed. a -> IO Bool: check if the resource is still alive. m b: the result of running your given action with a new pool. The key to understanding all of this is looking backwards from the result. 1) The result is m b. Where did this come from? 2) Well, there's a 'Pool a -> m b'. So it runs the action for me. So this is like a 'withFile :: FilePath -> (Handle -> IO a) -> IO a' function. 3) So if it is a 'Pool a', then the resource is of type 'a'. 4) The only function returning 'a' is 'IO a', so this creates the resource. 5) By the same reason, 'a -> IO ()' disposes it. 6) 'a -> IO Bool' is confusing, but there's a doc for that. 7) 'Int' is also confusing, but since this is a pool, this must be the maximum number of open resources. HTH, -- Felipe.

On 07/11/11 11:41, Felipe Almeida Lessa wrote:
Yes, there's a documentation problem. But you can guess by the types =).
IO a: opens a new resource a. a -> IO (): disposes the given resource. Int: maximum number of open resources. Pool a -> m b: the action you want to execute; after it executes, the pool is destroyed. a -> IO Bool: check if the resource is still alive. m b: the result of running your given action with a new pool.
The key to understanding all of this is looking backwards from the result.
1) The result is m b. Where did this come from? 2) Well, there's a 'Pool a -> m b'. So it runs the action for me. So this is like a 'withFile :: FilePath -> (Handle -> IO a) -> IO a' function. 3) So if it is a 'Pool a', then the resource is of type 'a'. 4) The only function returning 'a' is 'IO a', so this creates the resource. 5) By the same reason, 'a -> IO ()' disposes it. 6) 'a -> IO Bool' is confusing, but there's a doc for that. 7) 'Int' is also confusing, but since this is a pool, this must be the maximum number of open resources.
Magic, got it =)

On Mon, Jul 11, 2011 at 6:27 PM, Michael Orlitzky
On 07/10/11 15:37, Felipe Almeida Lessa wrote:
You don't need to do it, it is already done =). See the pool package by Michael Snoyman on Hackage [1]. More specifically, see createPoolCheckAlive [2].
Cheers,
[1] http://hackage.haskell.org/package/pool [2] http://hackage.haskell.org/packages/archive/pool/0.1.0.2/doc/html/Data-Pool....
How do people use this stuff? The README is empty, there's no documentation, and none of the functions are commented. I'm forced to conclude that there are people capable of looking at this,
createPoolCheckAlive :: MonadControlIO m => IO a -> (a -> IO ()) -> Int -> (Pool a -> m b) -> (a -> IO Bool) -> m b
and knowing immediately what it does.
I'm still a beginner, and I don't think I have a use for this package (what does it do?), but I've run into similar situations with other packages and would genuinely like to know (as opposed to just complain about it).
Author of the package speaking. I agree, that's a problem :). I've just uploaded a new version[1] that is properly documented, though it will take a bit for Hackage to generates the Haddocks. As Brandon said, the reason this wasn't documented in the first place is that it was an internally used module for Persistent that I decided after the fact to release separately. As for Bryan's resource-pool: currently I would strongly recommend *against* using it for any purpose. It is based on MonadCatchIO-transformers[2], which is a subtly broken package. In particular, when I tried using it for pool/persistent in the first place, I ended up with double-free bugs from SQLite. As an abridged history, I ended up writing a replacement called MonadInvertIO, which later was superceded by MonadPeelIO[3] and MonadControlIO[4]. The latter two packages are both much more correct that MonadCatchIO, and either one should be used in its place. I did email Bryan about this a bit ago, but he didn't get back (he *is* a very busy guy). Ideally, I think that we don't need to have both pool and resource-pool, but such is the situation right now. Michael [1] http://hackage.haskell.org/package/pool-0.1.0.3 [2] http://hackage.haskell.org/package/MonadCatchIO-transformers-0.2.2.2 [3] http://hackage.haskell.org/package/monad-peel [4] http://hackage.haskell.org/package/monad-control-0.2.0.1

On 07/11/11 13:49, Michael Snoyman wrote:
Author of the package speaking. I agree, that's a problem :). I've just uploaded a new version[1] that is properly documented, though it will take a bit for Hackage to generates the Haddocks. As Brandon said, the reason this wasn't documented in the first place is that it was an internally used module for Persistent that I decided after the fact to release separately.
Thanks for humoring me. I guess I'll begin tracking down other authors and making them feel guilty!

On Mon, Jul 11, 2011 at 10:49 AM, Michael Snoyman
I did email Bryan about this a bit ago, but he didn't get back [...]
Thanks for jogging my memory. I've released an updated version of resource-pool that drops the dependency on that package. http://hackage.haskell.org/package/resource-pool

12 July 2011 05:49, Michael Snoyman
As for Bryan's resource-pool: currently I would strongly recommend *against* using it for any purpose. It is based on MonadCatchIO-transformers[2], which is a subtly broken package. In particular, when I tried using it for pool/persistent in the first place, I ended up with double-free bugs from SQLite.
Do you have a reference explaining this brokenness? e.g. a mailing list message? I wasn't aware of this. Are the other MonadCatchIO-* packages also broken? Alistair

On Jul 11, 2011, at 10:48 PM, Alistair Bayley wrote:
12 July 2011 05:49, Michael Snoyman
wrote: As for Bryan's resource-pool: currently I would strongly recommend *against* using it for any purpose. It is based on MonadCatchIO-transformers[2], which is a subtly broken package. In particular, when I tried using it for pool/persistent in the first place, I ended up with double-free bugs from SQLite.
Do you have a reference explaining this brokenness? e.g. a mailing list message? I wasn't aware of this. Are the other MonadCatchIO-* packages also broken?
http://www.haskell.org/pipermail/haskell-cafe/2010-October/084890.html

On Sun, Jul 10, 2011 at 15:04, Richard Wallace
What I'd like to do is something a bit smarter. When a request to the SOAP server is to be made, if we have a token then we try and use it. If it fails, we reauthenticate and get a new token. When establishing a new token, other threads trying to send SOAP requests should block until a new token is available. There are other conditions that
Typically you'd use a Chan or TChan for communication with the SOAP thread. Other threads write to the Chan and block if the reading thread is busy. There's also a non-blocking version. (TChan uses STM, which provides atomic transactions; in this case it may be overkill, especially if you're just getting started.) I wouldn't be surprised if there's already a (or several) package on Hackage for it.... -- brandon s allbery allbery.b@gmail.com wandering unix systems administrator (available) (412) 475-9364 vm/sms

Have you looked at Bryan O'Sullivan's resource-pool[1] library? I've used it with MySQL, Redis and Riak connection pooling. It's been robust and easy to use. It seems like it would be a good fit for your application. Cheers, Ozgun [1] http://hackage.haskell.org/package/resource-pool On Sun, Jul 10, 2011 at 3:04 PM, Richard Wallace < rwallace@thewallacepack.net> wrote:
Hello all,
I'm trying to understand how to properly structure a connection to an external system. I'm writing an application that processes requests (it's an IRC bot - ya, I've looked at lambabot but it's a bit beyond my current understanding and I'm really trying to learn this stuff and find the best way to do that is to write it myself) and sends requests to an external system, via SOAP. The SOAP requests should be sent on separate threads. The SOAP server requires an initial request for authentication be made to obtain a token used on subsequent requests.
There are three ways I can structure this. One is to do the authentication with the SOAP server when the application starts up and just use that token for all subsequent requests. This will fall over if the token times out and I'd have to restart the application. Another way is to do authenticate with the server and get a new token for each request. This is obviously really inefficient.
What I'd like to do is something a bit smarter. When a request to the SOAP server is to be made, if we have a token then we try and use it. If it fails, we reauthenticate and get a new token. When establishing a new token, other threads trying to send SOAP requests should block until a new token is available. There are other conditions that should be handled - such as an exponential back-off when the SOAP server can't be reached - but I feel like reestablishing the authentication token is really the key concern here. Unfortunately, I have no idea right now how to write this in Haskell.
Any pointers would be awesome. Thanks!
Rich
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
participants (9)
-
Alistair Bayley
-
Brandon Allbery
-
Bryan O'Sullivan
-
Daniel Gorín
-
Felipe Almeida Lessa
-
Michael Orlitzky
-
Michael Snoyman
-
Ozgun Ataman
-
Richard Wallace