A rule of thumb that has served me well w.r.t exposing internal modules is to expose a Data.Foo.Internal but make it clear it is a very fragile interface.
Even by going to far as to say this module does not follow the PVP and that they should expect breaking changes to come fast and often. Users should only safely depend on it with minor-version specific bounds then. This ameliorates the concerns about how it ties your hands as an implementor.
Breaking this API shouldn't require discussion on the mailing lists, as it is an internal implementation detail. This should further ameliorate concerns about it tying your hands as an implementor.
This lets users who need to write performant code not have to fork the entire package. (I've had to do this with Map before, Text and other packages that have been rather hide-bound about not exposing implementation details, it sucks.)
My experience is maybe 1-2% of your users need it, but when they need it it is the difference between the package being usable or having to be completely replaced with something else. They are also the kind of users who understand the need for the best possible implementation and who will roll with the punches.
Chasing after changes in the implementation is generally far less work than maintaining an entire fork.
-Edward