pool: Why doesn't it block?

Hello Michael, hello fellow haskellers, there is something, which has bothered me for quite a while, but now it has become a serious problem for me, because I see it as a bug, and there is no elegant way to work around it. I wonder if it's the right semantics for Data.Pool to simply fail with an exception, if the pool is exhausted. It would be much more appropriate, if it would just block, until a resource becomes available. Otherwise it's just /safe/ for multi-threading, but not really /useful/ for it. I noticed this when I launched 512 worker threads, but my pool had only 16 database connections. I need the pool to block, until a resource is available. It's also common that my Yesod site just returns an internal server error, when the pool is exhausted, so you can only handle as many connections successfully as there are database connections. I would expect Yesod to wait for a connection to become available instead of simply blowing the request. Blocking should at least be an option and be somehow reachable from Yesod/persistent. Greets, Ertugrul -- nightmare = unsafePerformIO (getWrongWife >>= sex) http://ertes.de/

The documentation seems to indicate that the behaviour should be blocking, so if it's not, might be a bug.
withResource :: MonadCatchIO m => Pool a -> (a -> m b) -> m b Temporarily take a resource from a Pool, perform an action with it, and return it to the pool afterwards. .... * If the maximum number of resources has been reached, this function blocks until a resource becomes available.
http://hackage.haskell.org/packages/archive/resource-pool/0.1.1.0/doc/html/D... Cheers, Edward Excerpts from Ertugrul Soeylemez's message of Sun Jun 12 03:47:37 -0400 2011:
Hello Michael, hello fellow haskellers,
there is something, which has bothered me for quite a while, but now it has become a serious problem for me, because I see it as a bug, and there is no elegant way to work around it.
I wonder if it's the right semantics for Data.Pool to simply fail with an exception, if the pool is exhausted. It would be much more appropriate, if it would just block, until a resource becomes available. Otherwise it's just /safe/ for multi-threading, but not really /useful/ for it.
I noticed this when I launched 512 worker threads, but my pool had only 16 database connections. I need the pool to block, until a resource is available.
It's also common that my Yesod site just returns an internal server error, when the pool is exhausted, so you can only handle as many connections successfully as there are database connections. I would expect Yesod to wait for a connection to become available instead of simply blowing the request.
Blocking should at least be an option and be somehow reachable from Yesod/persistent.
Greets, Ertugrul

Wrong package, that's resource-pool (which I wasn't aware of until
this moment). Ertugrul is referring to pool:
http://hackage.haskell.org/packages/archive/pool/0.1.0.2/doc/html/Data-Pool....
.
My original intention of splitting pool off from persistent was so
others could use it. If Bryan's maintaining resource-pool instead, I'd
have no problem deprecating pool and using resource-pool in its place.
Michael
On Sun, Jun 12, 2011 at 12:06 PM, Edward Z. Yang
The documentation seems to indicate that the behaviour should be blocking, so if it's not, might be a bug.
withResource :: MonadCatchIO m => Pool a -> (a -> m b) -> m b Temporarily take a resource from a Pool, perform an action with it, and return it to the pool afterwards. .... * If the maximum number of resources has been reached, this function blocks until a resource becomes available.
http://hackage.haskell.org/packages/archive/resource-pool/0.1.1.0/doc/html/D...
Cheers, Edward
Excerpts from Ertugrul Soeylemez's message of Sun Jun 12 03:47:37 -0400 2011:
Hello Michael, hello fellow haskellers,
there is something, which has bothered me for quite a while, but now it has become a serious problem for me, because I see it as a bug, and there is no elegant way to work around it.
I wonder if it's the right semantics for Data.Pool to simply fail with an exception, if the pool is exhausted. It would be much more appropriate, if it would just block, until a resource becomes available. Otherwise it's just /safe/ for multi-threading, but not really /useful/ for it.
I noticed this when I launched 512 worker threads, but my pool had only 16 database connections. I need the pool to block, until a resource is available.
It's also common that my Yesod site just returns an internal server error, when the pool is exhausted, so you can only handle as many connections successfully as there are database connections. I would expect Yesod to wait for a connection to become available instead of simply blowing the request.
Blocking should at least be an option and be somehow reachable from Yesod/persistent.
Greets, Ertugrul
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Oops, my bad! :-) Cheers, Edward Excerpts from Michael Snoyman's message of Sun Jun 12 05:15:41 -0400 2011:
Wrong package, that's resource-pool (which I wasn't aware of until this moment). Ertugrul is referring to pool: http://hackage.haskell.org/packages/archive/pool/0.1.0.2/doc/html/Data-Pool.... .
My original intention of splitting pool off from persistent was so others could use it. If Bryan's maintaining resource-pool instead, I'd have no problem deprecating pool and using resource-pool in its place.
Michael
On Sun, Jun 12, 2011 at 12:06 PM, Edward Z. Yang
wrote: The documentation seems to indicate that the behaviour should be blocking, so if it's not, might be a bug.
withResource :: MonadCatchIO m => Pool a -> (a -> m b) -> m b Temporarily take a resource from a Pool, perform an action with it, and return it to the pool afterwards. .... * If the maximum number of resources has been reached, this function blocks until a resource becomes available.
http://hackage.haskell.org/packages/archive/resource-pool/0.1.1.0/doc/html/D...
Cheers, Edward
Excerpts from Ertugrul Soeylemez's message of Sun Jun 12 03:47:37 -0400 2011:
Hello Michael, hello fellow haskellers,
there is something, which has bothered me for quite a while, but now it has become a serious problem for me, because I see it as a bug, and there is no elegant way to work around it.
I wonder if it's the right semantics for Data.Pool to simply fail with an exception, if the pool is exhausted. It would be much more appropriate, if it would just block, until a resource becomes available. Otherwise it's just /safe/ for multi-threading, but not really /useful/ for it.
I noticed this when I launched 512 worker threads, but my pool had only 16 database connections. I need the pool to block, until a resource is available.
It's also common that my Yesod site just returns an internal server error, when the pool is exhausted, so you can only handle as many connections successfully as there are database connections. I would expect Yesod to wait for a connection to become available instead of simply blowing the request.
Blocking should at least be an option and be somehow reachable from Yesod/persistent.
Greets, Ertugrul
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

On Sun, Jun 12, 2011 at 10:47 AM, Ertugrul Soeylemez
Hello Michael, hello fellow haskellers,
there is something, which has bothered me for quite a while, but now it has become a serious problem for me, because I see it as a bug, and there is no elegant way to work around it.
I wonder if it's the right semantics for Data.Pool to simply fail with an exception, if the pool is exhausted. It would be much more appropriate, if it would just block, until a resource becomes available. Otherwise it's just /safe/ for multi-threading, but not really /useful/ for it.
I noticed this when I launched 512 worker threads, but my pool had only 16 database connections. I need the pool to block, until a resource is available.
It's also common that my Yesod site just returns an internal server error, when the pool is exhausted, so you can only handle as many connections successfully as there are database connections. I would expect Yesod to wait for a connection to become available instead of simply blowing the request.
Blocking should at least be an option and be somehow reachable from Yesod/persistent.
Greets, Ertugrul
-- nightmare = unsafePerformIO (getWrongWife >>= sex) http://ertes.de/
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Well, there's withPoolAllocate, which will allocate a new resource if the pool is empty. I don't see a problem adding withPoolBlock as well. Using threadDelay, this is possible already: withPoolBlock pool f = do x <- withPool f maybe (liftIO (threadDelay timeout) >> withPoolBlock pool f) return x Though I'm sure we could do better than that. Michael
participants (3)
-
Edward Z. Yang
-
Ertugrul Soeylemez
-
Michael Snoyman