More fun with Haskell-in-the-large: linking time has become the main bottleneck in our development cycle. The standard solution would be to use an incremental linker, but it seems that gnu does not yet support this:-|
Hmm, I've never heard of linking being a bottleneck. Even GHC itself links in about 3-4 seconds here. One common problem is that linking on a network filesystem takes a *lot* longer than linking objects from a local disk. It's always a good idea to keep the build tree on the local disk, even if the sources are NFS-mounted.
I also have this problem, and while being on a local disk rather than NFS helps, it doesn't help all that much. For large projects, I usually have time to get a cup of coffee while linking (admittedly only four doors away, but...). When on NFS, I have time to go to the local coffeehouse...
Ok, it looks like we need to investigate this. NFS isn't the problem in itself: I realised that our GHC installation is NFS-mounted on the machine I tried the experiment on, and it makes very little difference (although Linux's NFS implementation is a bit fast & loose when it comes to caching, I seem to recall). Those who experience long link times (longer than a few seconds), please reply with your - platform / OS version - versions of relevent things (GHC, GCC, binutils). - time to link 'main = print "hello"'. Does starting up GHCi take a long time? Would someone like to strace (or truss, or ktrace or whatever) the ld process and see what it is doing for all that time. Is it CPU or I/O bound? Cheers, Simon
On Wed, Nov 27, 2002 at 03:55:54PM -0000, Simon Marlow wrote:
Those who experience long link times (longer than a few seconds), please reply with your
- platform / OS version - versions of relevent things (GHC, GCC, binutils). - time to link 'main = print "hello"'.
Platform: Debian unstable, with the Debian GHC package (maintained by Michael Weber, iirc) GHC version: 5.04 GCC version: 2.95.4 binutils: GNU ld version 2.13.90.0.10 20021010 Debian GNU/Linux For the runtime loader example, this is the thing which takes ages to link: ghc -package lang -optl-export-dynamic -fglasgow-exts -O2 -o TextFilter Main.o TextFilter.o TextFilterPluginAPI.o -ldl -lHSrts -lHSlang /usr/lib/ghc-5.04/HSbase.o /usr/lib/ghc-5.04/HSlang.o ../runtime_loader/libRuntimeLoader.a make 169.92s user 0.86s system 98% cpu 2:52.81 total Linking in a 'main = print "hello"' is normal: 15:13 exodus:~/bar % ghc Main.hs ghc Main.hs 3.44s user 0.23s system 78% cpu 4.677 total Here's the fun part: 15:18 exodus:~/bar % time ghc -package lang /usr/lib/ghc-5.04/HSbase.o /usr/lib/ghc-5.04/HSlang.o Main.hs ghc -package lang /usr/lib/ghc-5.04/HSbase.o /usr/lib/ghc-5.04/HSlang.o 1.42s user 0.34s system 105% cpu 1.666 total 15:30 exodus:~/bar % time ghc -optl-export-dynamic -package lang Main.hs ghc -optl-export-dynamic -package lang Main.hs 3.68s user 0.21s system 102% cpu 3.793 total 15:22 exodus:~/bar % time ghc -optl-export-dynamic -package lang /usr/lib/ghc-5.04/HSbase.o /usr/lib/ghc-5.04/HSlang.o Main.hs ghc -optl-export-dynamic -package lang /usr/lib/ghc-5.04/HSbase.o Main.hs 169.05s user 0.87s system 100% cpu 2:49.85 total I guess it looks like -optl-export-dynamic in conjunction with linking in the .o's is the killer. strace seems to indicate that ld isn't doing any syscalls when it's doing the -export-dynamic work; it just mmaps a huge wad of memory before starting, then it doesn't make a single syscall until it goes to write the final output. Presumably it's mmapping the .o's into memory to do the -export-dynamic magic.
Does starting up GHCi take a long time?
GHCI startup appears to be normal (a few seconds).
At least for me the culprit is -optl-export-dynamic ... I'm not
sure that this is in GHC's problem domain any more.
--
#ozone/algorithm
participants (2)
-
Andre Pang -
Simon Marlow