Hi. This is not strictly a GHC question. I observed that "ld" when linking GHC-compiled programs eats 0.5 GB of resident memory. ~3 times more than GHC or Haddock. So if there is not enough free memory "ld" uses virtual memory and works very slowly (practically does not use CPU). I'd like to know, is anyone interested in researching and improving memory usage? Is it "ld" or GHC problem? I found a similar discussion here http://article.gmane.org/gmane.comp.lang.haskell.cafe/66189 but it was too broad and went nowhere. -- Best regards, Roman Beslik.
| This is not strictly a GHC question. I observed that "ld" when linking | GHC-compiled programs eats 0.5 GB of resident memory. ~3 times more than | GHC or Haddock. So if there is not enough free memory "ld" uses virtual | memory and works very slowly (practically does not use CPU). I'd like to | know, is anyone interested in researching and improving memory usage? Is | it "ld" or GHC problem? I found a similar discussion here | http://article.gmane.org/gmane.comp.lang.haskell.cafe/66189 | but it was too broad and went nowhere. Roman, I think it would be great if you could find a group of people to look into this. Even if it's an ld bad-perf case, maybe GHC could do something to dodge ld's bad case. Simon
Roman Beslik wrote:
This is not strictly a GHC question. I observed that "ld" when linking GHC-compiled programs eats 0.5 GB of resident memory. ~3 times more than GHC or Haddock.
Oh I remember investigating this problem 2 years ago. Looking back it seems I never made my observations public. GNU ld uses hash tables to store the symbols of each object file. The default size of those hash tables is 4051, which is a huge waste in Haskell's split-objs case, where object files are plenty and only have a couple of dozen symbols (most of them external references) at most. I've been running binutils with this patch, http://int-e.home.tlink.de/haskell/binutils-2.18-tune-bfd-hash.patch (still applies to 2.20) which changes the default size to 31 and adds some more intermediate sizes. This improves memory usage a lot, at a small cost in running time (if I had numbers, they're lost by now). HTH, Bertram
Thank you very much. With your patch "ld" eats 3 times less resident memory when compiling Hackage packages. If someone here uses Arch Linux, I uploaded patched "binutils" to AUR with the name "binutils-tune-bfd-hash". On 17.04.10 16:59, Bertram Felgenhauer wrote:
I've been running binutils with this patch,
http://int-e.home.tlink.de/haskell/binutils-2.18-tune-bfd-hash.patch (still applies to 2.20)
-- Best regards, Roman Beslik.
You could also have a look at the new 'Gold' linker that's part of
binutils. Not sure if this has been tested before with ghc but its
meant to have far greater performance (in time is what I've heard, not
sure about memory) then 'ld' and be a drop in replacement.
On 17 April 2010 00:00, Roman Beslik
Hi. This is not strictly a GHC question. I observed that "ld" when linking GHC-compiled programs eats 0.5 GB of resident memory. ~3 times more than GHC or Haddock. So if there is not enough free memory "ld" uses virtual memory and works very slowly (practically does not use CPU). I'd like to know, is anyone interested in researching and improving memory usage? Is it "ld" or GHC problem? I found a similar discussion here http://article.gmane.org/gmane.comp.lang.haskell.cafe/66189 but it was too broad and went nowhere.
-- Best regards, Roman Beslik.
_______________________________________________ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
I've tried. First error was that "gold" does not recognize "-pthread" option. If I remove "-pthread", "gold" complains about many undefined references, "strlen" for instance. So, I did not succeed. I guess that GHC and "gold" need adjustments to work together. On 18.04.10 09:20, David Terei wrote:
You could also have a look at the new 'Gold' linker that's part of binutils.
-- Best regards, Roman Beslik.
participants (4)
-
Bertram Felgenhauer -
David Terei -
Roman Beslik -
Simon Peyton-Jones