<the.dead.shall.rise@gmail.com> wrote:
> Thanks. I'll look into how to optimise .hi loading by more traditional
> means, then.

Lennart is working on speeding up the binary package (which I believe
is used to decode the .hi files.) His work might benefit this effort.

Last time I tested it, mmap still offered better performance than fread on linux.  In addition to improving the deserialization code it would seem like a good idea to mmap the whole file at the outset as well.

It seems like readBinMem is the relevant function (readIFace -> readBinIFace -> readBinMem), which occurs here:

   https://github.com/ghc/ghc/blob/08894f96407635781a233145435a78f144accab0/compiler/utils/Binary.hs#L222

Currently it does one big hGetBuf to read the file.  Since the interface files aren't changing dynamically, I think it's safe to just replace this code with an mmap.

It's nice to see that we have several wrapped versions of mmap provided on hackage:

http://hackage.haskell.org/package/vector-mmap
http://hackage.haskell.org/package/bytestring-mmap-0.2.2
http://hackage.haskell.org/package/mmap-0.5.7

Cheers,
  -Ryan