This is probably widely known already but I just realized you can sort-of have closed classes as it is:
module SomeModule( SomeClass, someMethod ) where
class SomeClassInternal a where someMethodInternal :: a -> Integer
instance SomeClassInternal Integer where someMethodInternal = id
instance SomeClassInternal [a] where someMethodInternal = fromIntegral . length
instance SomeClassInternal Bool where someMethodInternal False = 0 someMethodInternal True = 42
class SomeClassInternal a => SomeClass a instance SomeClass Integer instance SomeClass [a] instance SomeClass Bool
someMethod :: SomeClass a => a -> Integer someMethod = someMethodInternal
Basically, not exporting a class is a very obvious way to prevent people from declaring new instances. The problem is that they also can't refer to it in any way, and if it shows up in the public API in a significant way (besides it being just weird) they have to try and manage their business without using type signatures. The solution is pretty simple: declare a private class with private methods, and a corresponding public class which implies the private one, with instances for all the same types, and have everything in the public API use the public class instead. That way people still can't declare new instances for the private class (it's not visible), nor for the public one (because they can't satisfy the superclass constraint), but otherwise they're free to use it in type signatures and do whatever they want. I don't think the compiler would use this to reason about the code in new ways (i.e. the typechecker would still treat it as open), but it's at least something. (You could also use UndecidableInstances and `instance SomeClassInternal a => SomeClass a` for less repetition and easier maintainability; the advantage of not doing so is that you don't need UndecidableInstances, and the resulting Haddocks are friendlier, especially if/when Haddock gets updated to hide instances of non-exported classes.)