On Wed, Jul 14, 2010 at 4:48 PM, Hackage
Comment(by duncan):
It should be straightforward to extend the size of the types used to cope with bigger tarballs. The only cost will be a bigger index. The reason for the limitations in the hackage code is simply to save space by keeping the indexes very compact.
Yup. My branch of cabal-install has this done: http://community.haskell.org/~aslatter/code/cabal-install/index/ which requires: http://community.haskell.org/~aslatter/code/tarindex/ This should be considered a prototype, as we would need to more carefully consider what dependencies we should be pulling in for this. For "cabal list $pkgname" I go from ~1s to ~550ms. For "cabal install --dry-run happstack" I go from ~2.5s to ~1.9s. According to GHC profiling most of the time spent in "cabal list" is in looking up paths in the tar-index offset store (this happens ~9000 times). I don't have profiling data through package boundaries for some reason, so there might yet be low-hanging fruit over there. Antoine