GHC exceeding command line length limit with split-objs - and a possible fix

Hi, I'm building gtk2hs on a mac with -fsplit-objs, and during compilation of the largest file, which produces about 7000 split objects, the linking phase fails. I'm assuming that's because the command line length has been exceeded, because other files compile fine, without -fsplit-objs it compiles fine too, and it compiles fine *with* -fsplit-objs on other OS - so perhaps the reason is in mac's tempfile name generation (they're longer than on other platforms) or something. Anyway. I think a nice fix would be to employ gcc's ability to read options from a file - gcc @file - and write overly long option strings into temp files. It'd be fun for me to implement this as my first contribution to ghc; is this a good idea in general? -- Eugene Kirpichov Principal Engineer, Mirantis Inc. http://www.mirantis.com/ Editor, http://fprog.ru/

Oh well... looks like building ghc won't be easy, as it doesn't build with
llvm-gcc and it's not easy to get a "real" gcc on Lion. But I don't stop
trying :)
On Wed, Jan 11, 2012 at 11:12 AM, Eugene Kirpichov
Hi,
I'm building gtk2hs on a mac with -fsplit-objs, and during compilation of the largest file, which produces about 7000 split objects, the linking phase fails. I'm assuming that's because the command line length has been exceeded, because other files compile fine, without -fsplit-objs it compiles fine too, and it compiles fine *with* -fsplit-objs on other OS - so perhaps the reason is in mac's tempfile name generation (they're longer than on other platforms) or something.
Anyway.
I think a nice fix would be to employ gcc's ability to read options from a file - gcc @file - and write overly long option strings into temp files.
It'd be fun for me to implement this as my first contribution to ghc; is this a good idea in general?
-- Eugene Kirpichov Principal Engineer, Mirantis Inc. http://www.mirantis.com/ Editor, http://fprog.ru/
-- Eugene Kirpichov Principal Engineer, Mirantis Inc. http://www.mirantis.com/ Editor, http://fprog.ru/

On Wed, Jan 11, 2012 at 02:12, Eugene Kirpichov
I think a nice fix would be to employ gcc's ability to read options from a file - gcc @file - and write overly long option strings into temp files.
What immediately occurs to me is, what if the problem (or another manifestation of same) occurs in the ld step? OS X's ld(1) doesn't have such an option /per se/, and binutils ld(1) does not reliably create valid Mach-O objects. I would consider "batching" split-objs files into static archives (see ar(1) and ranlib(1)). This also has the advantages of being portable (other platforms other have length limits; I believe it's the main reason split-objs is disabled by default on e.g. Solaris) and that with many linkers it's faster than individual objects because it can use the archive table of contents to speed up searching for files and symbols. -- brandon s allbery allbery.b@gmail.com wandering unix systems administrator (available) (412) 475-9364 vm/sms

Hi Brandon,
Thanks - looks like this would be a modification of the linking stage,
splitting it into two parts: batching objects and then actually linking
them.
Do you think that what I originally proposed is still a good thing to have
before implementing your solution? (it definitely would be for myself, as
it's easier to do and I'd then quicker become able to build my application
with split-objs and shrink its executable size from 11 to 2Mb)
On Wed, Jan 11, 2012 at 9:48 PM, Brandon Allbery
On Wed, Jan 11, 2012 at 02:12, Eugene Kirpichov
wrote: I think a nice fix would be to employ gcc's ability to read options from a file - gcc @file - and write overly long option strings into temp files.
What immediately occurs to me is, what if the problem (or another manifestation of same) occurs in the ld step? OS X's ld(1) doesn't have such an option /per se/, and binutils ld(1) does not reliably create valid Mach-O objects.
I would consider "batching" split-objs files into static archives (see ar(1) and ranlib(1)). This also has the advantages of being portable (other platforms other have length limits; I believe it's the main reason split-objs is disabled by default on e.g. Solaris) and that with many linkers it's faster than individual objects because it can use the archive table of contents to speed up searching for files and symbols.
-- brandon s allbery allbery.b@gmail.com wandering unix systems administrator (available) (412) 475-9364 vm/sms
-- Eugene Kirpichov Principal Engineer, Mirantis Inc. http://www.mirantis.com/ Editor, http://fprog.ru/

On Wed, Jan 11, 2012 at 14:50, Eugene Kirpichov
Do you think that what I originally proposed is still a good thing to have before implementing your solution? (it definitely would be for myself, as it's easier to do and I'd then quicker become able to build my application with split-objs and shrink its executable size from 11 to 2Mb)
Only if it works; my expectation is that all you will accomplish is to move the failure point, as the command line for ld is usually larger than the one for gcc. It is possible that gcc will do something to deal with it itself, but I have a suspicion that only works (if it does) with binutils ld, which is not usable on OS X. BTW, I'm actually suggesting that split-objs create an archive, where non-split-objs creates an object; from there I suspect you wouldn't actually need to handle anything any differently between the two cases (aside from specifying the .a instead of the .o as needed). -- brandon s allbery allbery.b@gmail.com wandering unix systems administrator (available) (412) 475-9364 vm/sms
participants (2)
-
Brandon Allbery
-
Eugene Kirpichov