On Wed, Jul 10, 2013 at 1:01 PM, John Lato <jwlato@gmail.com> wrote:

On Wed, Jul 10, 2013 at 5:02 PM, Erik Hesselink <hesselink@gmail.com> wrote:

On Wed, Jul 10, 2013 at 10:39 AM, John Lato <jwlato@gmail.com> wrote:
> I think 'shouldBeCaught' is more often than not the wrong thing. A
> whitelist of exceptions you're prepared to handle makes much more sense than
> excluding certain operations. Some common whitelists, e.g. filesystem
> exceptions or network exceptions, might be useful to have.

You'd think that, but there are common use cases. For example, if you
have a queue of work items, and a thread (or threads) processing them,
it is useful to catch all exceptions of these threads. You can then
log the exception, remove the item from the queue and put it in some
error bucket, and continue on to the next item. The same goes for e.g.
socket listening threads etc.

The thing here is that you are *not* actually handling the specific
exception, but instead failing gracefully. But you still want to be
able to kill the worker threads, and you don't want to handle
exceptions that you cannot recover from even by moving on to the next
work item.

I think that's a particularly niche use case. We have some similar code, and our approach is to have the thread re-throw (or terminate) after logging the exception. There's a separate thread that monitors the thread pool, and when threads die new ones are spawned to take their place (unless the thread pool is shutting down, of course). Spawning a new thread only happens on an exception and it's cheap anyway, so there's no performance issue.

As Haskell currently stands trying to sort out thread-control and fatal-for-real exceptions from other exceptions seems rather fiddly, unreliable, and prone to change between versions, so I think it's best avoided. If there were a standard library function to do it I might use it, but I wouldn't want to maintain it.

Maybe I'm just always working on niche cases then, because I run into this problem fairly regularly. Almost any time you want to write a library that will run code it doesn't entirely trust, this situation arises. Examples include:

Writing a web server (like Warp) which can run arbitrary user code. Warp must fail gracefully if the user code throws an exception, without bringing down the entire server thread.
Writing some kind of batch processing job which uses any library which may throw an exception. A white list approach would not be sufficient here, since we want to be certain that any custom exception types have been caught.
A system which uses worker threads to do much of its work. You want to make certain the worker threads don't unexpectedly die because some exception was thrown that you were not aware could be thrown. I use this technique extensively in Keter, and in fact some work I'm doing on that code base now is what triggered this email.

I think that, overall, Ertugrul's suggestion is probably the right one: we should be including richer information in the `Exception` typeclass so that there's no guessing involved, and any custom exception types can explicitly state what their recovery preference is. In the meanwhile, I think we could get pretty far by hard-coding some rules about standard exception types, and making an assumption about all custom exception types (e.g., they should be caught by a "catch all exceptions" call).

If we combine these two ideas, we could have a new package on Hackage which defines the right set of tags and provides a `tagsOf` function which works on any instance of Exception, which uses the assumptions I mentioned in the previous paragraph. If it's then decided that this is generally useful enough to be included in the Exception typeclass, we have a straightforward migration path:

Add the new method to the Exception typeclass, with a default implementation that conforms with our assumptions.
For any of the special standard exception types (e.g., AsyncException), override that default implementation.
Modify the external package to simply re-export the new method when using newer versions of base, using conditional compilation.
Any code written against that external package would work with both current and future versions of base.
The only incompatibility would be if someone writes code which overrides the typeclass method; that code would only work with newer bases, not current ones.

Any thoughts on this? I'm not sure exactly what would be the right method to add to the Exception typeclass, but if we can come to consensus on that and there are no major objections to my separate package proposal, I think this would be something moving forward on, including a library proposal.

Michael