Still trying to build unregisterised for FreeBSD/amd64
Hi, I have made some progress in compiling ghc-6.6 unregisterised for freebsd/amd64, but I am stuck again. To get the build on the host (freebsd/i386) to go through, I had to fix up Linker.c so that it would build. (Linker.c seems to define at least one utility function required by the rts, markRootPtrTable.) The instruction on the wiki to, "Don't worry if the build falls over in the RTS, we don't need the RTS yet," seems incorrect for ghc-6.6 at least. If the build fails in the rts, trying to proceed further just leads to more errors. I fixed up Linker.c by replacing the calls to mmap (which will need fixing up anyway on FreeBSD/amd64) with MAP_FAILED and copying the definitions of four relocation types for X86_64 from machine/elf.h on the target into do_Elf_Rela_relocations. (The build on the host complained that R_X86_64_64, R_X86_64_PC32, R_X86_64_32 and R_X86_64_32S were undefined.) With this change, the build on the host was able to complete. Collecting the .hc files works too, but the Makefile is out of date and references a few files that no longer exist. With a little hand editing, this is fixed and I have what I believe is a good set of .hc files. Over on the target, I unpack the fresh ghc-6.6-src.tar.bz2 and drop the .hc files on top of it. On FreeBSD, gmp is usually installed by the ports system in / usr/local/lib, so I preface invocations of configure in distrib/hc-build by CFLAGS="-I/usr/local/include" LDFLAGS="-L/usr/local/lib" so that the installed gmp is picked up in the configuration. I also edit mk/bootstrap.mk to add -L/usr/local/lib to HC_LD_BOOT_OPTS, and also add -lHSregex-compat -lHSregex-posix -lHSregex-base to HC_BOOT_LIBS. The directories to find the regex libraries also get added to HC_BOOT_LD_OPTS (-L$(FPTOOLS_TOP_ABS)/libraries/regex-*). I also have to delete cabal-setup from the SUBDIRS in libraries/Cabal/ Makefile. Otherwise the build fails complaining that there is no rule to make Cabal-setup.o. I also have to change libraries/base/Makefile to not try to build Prim.hs. It seems that when booting from .hc files, PrimopWrappers.hs does not in fact depend on Prim.hs; instead, PrimopWrappers.hs is not really used since we have PrimopWrappers.hc copied over from the host. (Without this change the Makefile tries to invoke the yet unbuilt genprimopcode program to build Prim.hs.) With these changes, distrib/hc-build is ready to roll. It chugs along happily until it tries to build Linker.c (which I have copied over from the host, so it has the changes that I made there) and then ghc-inplace segfaults: ../compiler/ghc-inplace -optc-O -optc-Wall -optc-W -optc-Wstrict- prototypes -optc-Wmissing-prototypes -optc-Wmissing-declarations - optc-Winline -optc-Waggregate-return -optc-Wbad-function-cast -optc- I../includes -optc-I. -optc-Iparallel -optc-DCOMPILING_RTS -optc- fomit-frame-pointer -optc-fno-strict-aliasing -H16m -O -optc-O2 - static -I. -#include HCIncludes.h -fvia-C -dcmm-lint -c Linker.c - o Linker.o Segmentation fault (core dumped) gmake: *** [Linker.o] Error 139 gmake: Leaving directory `/tmp/ghc-6.6/rts' I noticed that a year and a half ago when Wilhelm Kloke was porting ghc-6.4.1 to freebsd/amd64 he remarked that Linker.c had to be deleted. If I delete it from the target, the build just fails earlier, when genapply is being built, because of missing symbols: ------------------------------------------------------------------------ == gmake all -wr; in /tmp/ghc-6.6/utils/genapply ------------------------------------------------------------------------ gcc -x c GenApply.hc -o GenApply.o -c -O -fomit-frame-pointer -fno- asynchronous-unwind-tables -fno-unit-at-a-time -fno-builtin -DNO_REGS -DUSE_MINIINTERPRETER -D__GLASGOW_HASKELL__=606 -O -I/tmp/ghc-6.6/ includes -I/tmp/ghc-6.6/libraries/base/include -I/tmp/ghc-6.6/ libraries/unix/include -I/tmp/ghc-6.6/libraries/parsec/include -I/ tmp/ghc-6.6/libraries/readline/include -I. `echo | sed 's/^$/- DSTOLEN_X86_REGS=4/'` gcc -o genapply -fomit-frame-pointer -fno-asynchronous-unwind-tables -fno-unit-at-a-time -fno-builtin -DNO_REGS -DUSE_MINIINTERPRETER - D__GLASGOW_HASKELL__=606 -O -I/tmp/ghc-6.6/includes -I/tmp/ghc-6.6/ libraries/base/include -I/tmp/ghc-6.6/libraries/unix/include -I/tmp/ ghc-6.6/libraries/parsec/include -I/tmp/ghc-6.6/libraries/readline/ include -L/tmp/ghc-6.6/rts -L/tmp/ghc-6.6/libraries/base -L/tmp/ ghc-6.6/libraries/base/cbits -L/tmp/ghc-6.6/libraries/haskell98 -L/ tmp/ghc-6.6/libraries/parsec -L/tmp/ghc-6.6/libraries/regex-base -L/ tmp/ghc-6.6/libraries/regex-compat -L/tmp/ghc-6.6/libraries/regex- posix -L/tmp/ghc-6.6/libraries/Cabal -L/usr/local/lib -L/tmp/ghc-6.6/ libraries/template-haskell -L/tmp/ghc-6.6/libraries/readline -L/tmp/ ghc-6.6/libraries/unix -L/tmp/ghc-6.6/libraries/unix/cbits -u "GHCziBase_Izh_static_info" -u "GHCziBase_Czh_static_info" -u "GHCziFloat_Fzh_static_info" -u "GHCziFloat_Dzh_static_info" -u "GHCziPtr_Ptr_static_info" -u "GHCziWord_Wzh_static_info" -u "GHCziInt_I8zh_static_info" -u "GHCziInt_I16zh_static_info" -u "GHCziInt_I32zh_static_info" -u "GHCziInt_I64zh_static_info" -u "GHCziWord_W8zh_static_info" -u "GHCziWord_W16zh_static_info" -u "GHCziWord_W32zh_static_info" -u "GHCziWord_W64zh_static_info" -u "GHCziStable_StablePtr_static_info" -u "GHCziBase_Izh_con_info" -u "GHCziBase_Czh_con_info" -u "GHCziFloat_Fzh_con_info" -u "GHCziFloat_Dzh_con_info" -u "GHCziPtr_Ptr_con_info" -u "GHCziStable_StablePtr_con_info" -u "GHCziBase_False_closure" -u "GHCziBase_True_closure" -u "GHCziPack_unpackCString_closure" -u "GHCziIOBase_stackOverflow_closure" -u "GHCziIOBase_heapOverflow_closure" -u "GHCziIOBase_NonTermination_closure" -u "GHCziIOBase_BlockedOnDeadMVar_closure" -u "GHCziIOBase_Deadlock_closure" -u "GHCziWeak_runFinalizzerBatch_closure" -u "__stginit_Prelude" GenApply.o -lHSreadline -lHStemplate-haskell -lHSunix - lHSunix_cbits -lHSCabal -lHSregex-compat -lHSregex-posix -lHSregex- base -lHShaskell98 -lHSrts -lHSbase -lHSbase_cbits -lHSparsec -lgmp -lm /tmp/ghc-6.6/rts/libHSrts.a(GC.o)(.text+0x779): In function `GarbageCollect': : undefined reference to `markRootPtrTable' /tmp/ghc-6.6/libraries/base/libHSbase.a(Stable.o)(.text+0x360): In function `sxz_ret': : undefined reference to `hs_free_stable_ptr' /tmp/ghc-6.6/libraries/base/libHSbase.a(Typeable.o)(.text+0x2b85): In function `s4SA_ret': : undefined reference to `hs_free_stable_ptr' /tmp/ghc-6.6/libraries/base/libHSbase.a(Conc.o)(.text+0x3f7b): In function `base_GHCziConc_par_entry': : undefined reference to `newSpark' gmake[1]: *** [genapply] Error 1 gmake: *** [all] Error 1 gmake: Leaving directory `/tmp/ghc-6.6/utils' I am wondering what to try next. Could ghc-inplace on the target be crashing because of differences in say, ptr sizes on the host versus the target? Could I fool the host build into using 64 bit pointers by replacing explicit pointers with "long long" integers? Or should I try building a set of .hc files on a 64 bit linux host and use those to bootstrap the freebsd/amd64 target? Any help would be appreciated. Especially useful would be any comments on which parts of the instructions can be trusted and which may be bitrotted and needing revision. Best Wishes, Greg
Hi Gregory, On Thu, Mar 08, 2007 at 11:25:39AM -0500, Gregory Wright wrote:
I fixed up Linker.c by replacing the calls to mmap (which will need fixing up anyway on FreeBSD/amd64) with MAP_FAILED and copying the definitions of four relocation types for X86_64 from machine/elf.h on the target into do_Elf_Rela_relocations. (The build on the host complained that R_X86_64_64, R_X86_64_PC32, R_X86_64_32 and R_X86_64_32S were undefined.)
Anything that compiles should be fine until you want to get ghci working.
With this change, the build on the host was able to complete. Collecting the .hc files works too, but the Makefile is out of date and references a few files that no longer exist.
That sounds right - a couple of variants (e.g. debug) of one of the cmm files (AutoApply possibly) and one or two files that are in extralibs, if I remember correctly.
With a little hand editing, this is fixed and I have what I believe is a good set of .hc files.
Over on the target, I unpack the fresh ghc-6.6-src.tar.bz2 and drop the .hc files on top of it. On FreeBSD, gmp is usually installed by the ports system in / usr/local/lib, so I preface invocations of configure in distrib/hc-build by
CFLAGS="-I/usr/local/include" LDFLAGS="-L/usr/local/lib"
so that the installed gmp is picked up in the configuration. I also edit mk/bootstrap.mk to add -L/usr/local/lib to HC_LD_BOOT_OPTS, and also add
-lHSregex-compat -lHSregex-posix -lHSregex-base
to HC_BOOT_LIBS.
This sounds like you are using the original 6.6? You're probably better off trying to start from the 6.6 darcs branch instead (make a tarball of a checkout (might as well run autoreconf first) so you can be sure you have the same source on both machines). You should then be able to use --with-gmp-includes=/usr/local/include --with-gmp-libraries=/usr/local/lib and the HC_BOOT_LIBS should already be fixed.
I also have to delete cabal-setup from the SUBDIRS in libraries/Cabal/ Makefile. Otherwise the build fails complaining that there is no rule to make Cabal-setup.o.
*nod*
I also have to change libraries/base/Makefile to not try to build Prim.hs.
Again, fixed in the branch.
With these changes, distrib/hc-build is ready to roll. It chugs along happily until it tries to build Linker.c (which I have copied over from the host, so it has the changes that I made there) and then ghc-inplace segfaults:
../compiler/ghc-inplace -optc-O -optc-Wall -optc-W -optc-Wstrict- prototypes -optc-Wmissing-prototypes -optc-Wmissing-declarations - optc-Winline -optc-Waggregate-return -optc-Wbad-function-cast -optc- I../includes -optc-I. -optc-Iparallel -optc-DCOMPILING_RTS -optc- fomit-frame-pointer -optc-fno-strict-aliasing -H16m -O -optc-O2 - static -I. -#include HCIncludes.h -fvia-C -dcmm-lint -c Linker.c - o Linker.o Segmentation fault (core dumped) gmake: *** [Linker.o] Error 139 gmake: Leaving directory `/tmp/ghc-6.6/rts'
../compiler/ghc-inplace has successfully compiled other files by this point, right? It probably won't help, but what does it say if you add -v? I suspect the next step is to try gdb and see what it says, though. It might be necessary to repeat the build with -g -O0 in GhcHcOpts or other variables (be wary of hc-build trampling changes you make to mk/build.mk). It might even be useful to tweak the process so that you end up with a -debug ghc (in fact, this should probably be the default). I don't have instructions for how to do this, but I've done it in the past and don't remember any major difficulties. Basically stick "SRC_HC_OPTS += -debug" in compiler/Makefile and fix any problems that come up (you might need to do things like "make WAY=debug" in rts/, and modify the list of files that get put in the tarball). If you get stuck, please shout.
I am wondering what to try next. Could ghc-inplace on the target be crashing because of differences in say, ptr sizes on the host versus the target?
It's possible; the closer you can make the host and target the fewer problems you are likely to have.
Any help would be appreciated. Especially useful would be any comments on which parts of the instructions can be trusted and which may be bitrotted and needing revision.
I think they should pretty much work with the 6.6 branch, although I'm also not convinced by the "Don't worry if the build falls over in the RTS, we don't need the RTS yet." comment. Thanks Ian
Hi Ian, I have made some progress. Today I got 6.4.2 to build unregisterized on my FreeBSD/amd64 box. I went back to 6.4.2 because the only actual report of success I could find record of was Wilhelm Kloke's. He used 6.4.1. I made certain I had exactly the same versions of readline and gmp on both machines. This might not be necessary, but it's one less thing that might go wrong. I also downgraded my ghc to 6.4.2 on the host i386 system. Building 6.4.2 using 6.6 runs into trouble with the change in the mutable array API. The 6.4.2 build now runs successfully to the end of the hc-build script. I haven't installed it, but tried to build an out of the box 6.6 with the 6.4.2 ghc-inplace. (I applied my patch to Linker.c so that the build wouldn't fail merely because of undefined symbols.) I also patched the mangler by adding "|freebsd" to rules for /^x86_64-.*-(linux|openbsd)$/. The interesting thing is that out of the box using the 6.4.2 bootstrap compiler, 6.6 still crashes when compiling Linker.c. Not just a little crash; it hung the whole system. At that point it is using the ghc-inplace of 6.6. On Mar 13, 2007, at 3:05 PM, Ian Lynagh wrote:
This sounds like you are using the original 6.6? You're probably better off trying to start from the 6.6 darcs branch instead (make a tarball of a checkout (might as well run autoreconf first) so you can be sure you have the same source on both machines).
As I said, I'm using out of the box 6.6. Should I try a 6.6 from darcs?
../compiler/ghc-inplace -optc-O -optc-Wall -optc-W -optc-Wstrict- prototypes -optc-Wmissing-prototypes -optc-Wmissing-declarations - optc-Winline -optc-Waggregate-return -optc-Wbad-function-cast -optc- I../includes -optc-I. -optc-Iparallel -optc-DCOMPILING_RTS -optc- fomit-frame-pointer -optc-fno-strict-aliasing -H16m -O -optc-O2 - static -I. -#include HCIncludes.h -fvia-C -dcmm-lint -c Linker.c - o Linker.o Segmentation fault (core dumped) gmake: *** [Linker.o] Error 139 gmake: Leaving directory `/tmp/ghc-6.6/rts'
../compiler/ghc-inplace has successfully compiled other files by this point, right?
Yes. I've built the 6.6 compiler with the 6.4.2 bootstrap, and the inplace 6.6 has built everything up until Linker.c. Then crash, burn, smoking ruin.
It probably won't help, but what does it say if you add -v?
I suspect the next step is to try gdb and see what it says, though. It might be necessary to repeat the build with -g -O0 in GhcHcOpts or other variables (be wary of hc-build trampling changes you make to mk/build.mk).
It might even be useful to tweak the process so that you end up with a -debug ghc (in fact, this should probably be the default). I don't have instructions for how to do this, but I've done it in the past and don't remember any major difficulties. Basically stick "SRC_HC_OPTS += - debug" in compiler/Makefile and fix any problems that come up (you might need to do things like "make WAY=debug" in rts/, and modify the list of files that get put in the tarball).
I can add -v. Should I try to build a vanilla 6.6 with debugging or one out of darcs? Which would be more useful? Thanks for your help.
Thanks Ian
Best Wishes, Greg
Hi Gregory, On Tue, Mar 13, 2007 at 05:18:42PM -0400, Gregory Wright wrote:
The 6.4.2 build now runs successfully to the end of the hc-build script.
Ah, good, that'll hopefully make getting 6.6 working less painful.
The interesting thing is that out of the box using the 6.4.2 bootstrap compiler, 6.6 still crashes when compiling Linker.c. Not just a little crash; it hung the whole system.
Hmm. Do you know if it was eating all your memory?
As I said, I'm using out of the box 6.6. Should I try a 6.6 from darcs?
Yes, that's probably the best way to go. Put: GhcUnregisterised=YES GhcWithNativeCodeGen=NO GhcWithInterpreter=NO SplitObjs=NO GhcWithSMP=NO in mk/build.mk before you start building. If that breaks in the same way then set the -debug flag as I described before, delete compiler/stage1/ghc-6.whatever.it.is, run "make stage=1" in compiler/ (should relink ghc with -debug) and then try the failing command again. Assuming it still breaks, try gdb'ing it (note that it's a shell script that's being run, so you'll need to run the real binary and pass the extra args to it manually with gdb). Thanks Ian
Ian Lynagh wrote:
I think they should pretty much work with the 6.6 branch, although I'm also not convinced by the "Don't worry if the build falls over in the RTS, we don't need the RTS yet." comment.
That may well not be true, feel free to replace it with something more accurate. I haven't tried a bootstrap for a while. Cheers, Simon
participants (3)
-
Gregory Wright -
Ian Lynagh -
Simon Marlow