I agree that how the exception was thrown is more interesting than the type. I feel like there should be a way to express the necessary information via the type system, but I'm not convinced it's easy (or even possible).
Another issue to be aware of is that exceptions can be thrown from pure code, so if you don't fully evaluate your return value an exception can be thrown later, outside the catch block. In practice this usually means an NFData constraint, or some other constraint for which you can guarantee evaluation.
In the past I've been pretty vocal about my opposition to exceptions. It's still my opinion that they do not make it easy to reason about exceptional conditions. Regardless, as Haskell has them and uses them, I'd like to see improvements if possible. So if anyone is exploring the design space, I'd be willing to participate.