I'd be open to merging the existing Data.Functor.Contravariant module into base (inverting dependencies as needed), while leaving the remainder of the contravariant package intact. This module contains the class and a couple of example contravariant functors, much in the same vein as Data.Monoid. This would leave the more exotic machinery for 'contravariant applicatives' and all the generic programming bits in the package.
This would be similar to the migration of Data.Proxy from tagged into base, which left behind Data.Tagged or the migration of Data.Semigroup from semigroups, which left behind the generics module.
Almost all of the dependencies that would have inversions are packages I maintain personally. The only package that would require coordination with another maintainer would be inverting the instances for data types supplied by transformers.
If there was a sufficient call for it, I'd be open to moving the rest into base, but I don't anticipate a great deal of demand and this could always happen at a later date or as part of another proposal, should this change.
-Edward