Bootstrapping for Leopard
Dear All, I'm trying toupdate the Portfile so that ghc-6.8.2 can be built using MacPorts (handy for installing other ghc-dependent material from MacPorts like gtk2hs). The problem seems to be that the available bootstrap binary http://www.haskell.org/ghc/dist/6.6/ghc-6.6-i386-apple-darwin-bootstrap.tar.... causes a segfault on Leopard (below are the steps I took in my attempt to build ghc with this bootstrap binary). Using Manuel Chakravarty's binary distribution of ghc-6.8.1-i386-apple-darwin building ghc-6.8.2 works fine. It seemed to me that the easiest way to fix the bootstrap problem is to boot from C as described on the wiki (http://hackage.haskell.org/trac/ghc/wiki/Building/Porting). Two things were wrong, however. make hc-file-bundle Making the hc-file-bundle target failed, because not all rts/*.cmm had rts/*.hc counterparts after the build. The make fails because of this fragment from the Makefile (part of the hc-file-bundle target, starting from line 513): for f in `$(FIND) ghc-$(ProjectVersion)/compiler ghc-$(ProjectVersion)/rts -name "*.cmm" -follow -print` ""; do \ if test "x$$f" !=3D "x"; then \ echo `echo "$$f" | sed 's/cmm$$/hc/g' ` >> hc-files-to-go ; \ fi; \ done; This adds some non-existing .hc files to the hc-files-to-go list that tar will not find, later on, causing an error. I just added a test to see whether the file exists. If it does not, I call make for that .hc file explicitly, which solves the problem for most files. The files that don't have a make target, I simply omitted from the hc-files-to-go. I haven't been able to test the severity of this, because of the second problem. sh -e ./distrib/hc-build --prefix=3D$HOME/src/testbuild [--enable-hc-boot [--enable-hc-boot-unregisterised]] As a minor bug wrt. the wiki page, the hc-build is not executable by default. The more serious problem seems to be that the cold-boot option of configure is broken, because it ends in the error: checking for ld... /usr/bin/ld checking for path to top of build tree... ./configure: line 2651: -v0: command not found ./configure: line 2655: utils/pwd/pwd: No such file or directory configure: error: cannot determine current directory Upon inspection of the configure script, I found out that line 2651 uses the variable designating the ghc compiler. Are these bugs or am I messing things up? Regards, Philip
Philip K.F. Hölzenspies:
I'm trying toupdate the Portfile so that ghc-6.8.2 can be built using MacPorts (handy for installing other ghc-dependent material from MacPorts like gtk2hs). The problem seems to be that the available bootstrap binary http://www.haskell.org/ghc/dist/6.6/ghc-6.6-i386-apple-darwin-bootstrap.tar.... causes a segfault on Leopard (below are the steps I took in my attempt to build ghc with this bootstrap binary).
Using Manuel Chakravarty's binary distribution of ghc-6.8.1-i386- apple-darwin building ghc-6.8.2 works fine. It seemed to me that the easiest way to fix the bootstrap problem is to boot from C as described on the wiki (http://hackage.haskell.org/trac/ghc/wiki/Building/Porting). Two things were wrong, however.
make hc-file-bundle
Making the hc-file-bundle target failed, because not all rts/*.cmm had rts/*.hc counterparts after the build. The make fails because of this fragment from the Makefile (part of the hc-file-bundle target, starting from line 513):
for f in `$(FIND) ghc-$(ProjectVersion)/compiler ghc-$(ProjectVersion)/rts -name "*.cmm" -follow -print` ""; do \ if test "x$$f" !=3D "x"; then \ echo `echo "$$f" | sed 's/cmm$$/hc/g' ` >> hc-files-to-go ; \ fi; \ done;
This adds some non-existing .hc files to the hc-files-to-go list that tar will not find, later on, causing an error. I just added a test to see whether the file exists. If it does not, I call make for that .hc file explicitly, which solves the problem for most files. The files that don't have a make target, I simply omitted from the hc-files-to-go. I haven't been able to test the severity of this, because of the second problem.
I am not sure what the problem is here. Simon? Ian?
checking for ld... /usr/bin/ld checking for path to top of build tree... ./configure: line 2651: - v0: command not found ./configure: line 2655: utils/pwd/pwd: No such file or directory configure: error: cannot determine current directory
Upon inspection of the configure script, I found out that line 2651 uses the variable designating the ghc compiler.
This is due to a change of the configure stage that AFAIK was made to easy building on windows. Instead, of using shell commands/scripts (as GHC did previously) to obtain some configuration information (here the file path at which the top of the GHC build tree is located), the build system now uses small Haskell programs/scripts. This makes the build more portable ** if there is already a Haskell compiler on the system **. It however messes everything up if you want to build from HC files (and so don't have a Haskell compiler). I am not sure whether anybody thought about this problem when the change to using Haskell scripts instead of shell scripts was made. Ian? Simon? The only solution that I see is to replace the Haskell scripts by vanilla shell scripts in HC bundles. Even if that causes problems on windows (without cygwin), it would make HC bundles viable on Unix systems again. Manuel
On Wednesday 30 January 2008 10:13:01 Manuel M T Chakravarty wrote:
Upon inspection of the configure script, I found out that line 2651 uses the variable designating the ghc compiler.
This is due to a change of the configure stage that AFAIK was made to easy building on windows. Instead, of using shell commands/scripts (as GHC did previously) to obtain some configuration information (here the file path at which the top of the GHC build tree is located), the build system now uses small Haskell programs/scripts. This makes the build more portable ** if there is already a Haskell compiler on the system **. It however messes everything up if you want to build from HC files (and so don't have a Haskell compiler). I am not sure whether anybody thought about this problem when the change to using Haskell scripts instead of shell scripts was made. Ian? Simon?
The only solution that I see is to replace the Haskell scripts by vanilla shell scripts in HC bundles. Even if that causes problems on windows (without cygwin), it would make HC bundles viable on Unix systems again.
Dear Manuel, Your explanation makes perfect sense. For those more involved in the build process of GHC: How hard is it to roll back the vanilla shell scripts that once upon a time were used and to have 'configure' default to them when the --enable-hc-boot switch is used? Wrt. to the missing .hc files from rts/*.cmm: Since these weren't built in the make process, are they actually required? If not, it should simply be an extra test in the Makefile for the hc-file-bundle. Regards, Philip
On Wed, Jan 30, 2008 at 08:13:01PM +1100, Manuel M T Chakravarty wrote: [Philip K.F. Hölzenspies:]
make hc-file-bundle
Making the hc-file-bundle target failed, because not all rts/*.cmm had rts/*.hc counterparts after the build. The make fails because of this fragment from the Makefile (part of the hc-file-bundle target, starting from line 513):
for f in `$(FIND) ghc-$(ProjectVersion)/compiler ghc-$(ProjectVersion)/rts -name "*.cmm" -follow -print` ""; do \ if test "x$$f" !=3D "x"; then \ echo `echo "$$f" | sed 's/cmm$$/hc/g' ` >> hc-files-to-go ; \ fi; \ done;
This is strange. I've all kinds of trouble getting hc-bootstrapping back (for OpenBSD, but also in general), and I didn't see *that* problem.
checking for path to top of build tree... ./configure: line 2651: - v0: command not found ./configure: line 2655: utils/pwd/pwd: No such file or directory configure: error: cannot determine current directory [...] This is due to a change of the configure stage that AFAIK was made to easy building on windows. Instead, of using shell commands/scripts (as GHC did previously) to obtain some configuration information (here the file path at which the top of the GHC build tree is located), the build system now uses small Haskell programs/scripts. This makes the build more portable ** if there is already a Haskell compiler on the system **.
But it just doesn't make sense at all. You need a good set of shell commands at all, since they're used by configure as well as in Makefiles. I really can't believe that simple stuff like this doesn't work on windos: --- aclocal.m4.orig Mon Dec 10 19:11:31 2007 +++ aclocal.m4 Sun Jan 20 17:10:07 2008 @@ -1098,20 +1098,14 @@ AC_REQUIRE([AC_PROG_CC]) AC_DEFUN([FP_FIND_ROOT],[ AC_MSG_CHECKING(for path to top of build tree) -dnl This would be -dnl make -C utils/pwd clean && make -C utils/pwd -dnl except we don't want to have to know what make is called. Sigh. -if test ! -f utils/pwd/pwd && test ! -f utils/pwd/pwd.exe; then - cd utils/pwd - rm -f *.o - rm -f *.hi - rm -f pwd - rm -f pwd.exe - $WithGhc -v0 --make pwd -o pwd - cd ../.. -fi - -hardtop=`utils/pwd/pwd forwardslash` +case $HostPlatform in +*cygwin32|*mingw32) + hardtop=`pwd | tr \\ /` + ;; +*) + hardtop=`pwd` + ;; +esac if ! test -d "$hardtop"; then AC_MSG_ERROR([cannot determine current directory]) ifBuildable.hs is similar; it can be replaced by a shell script or even done within libraries/Makefile using very basic shell commands.
The only solution that I see is to replace the Haskell scripts by vanilla shell scripts in HC bundles. Even if that causes problems on windows (without cygwin), it would make HC bundles viable on Unix systems again.
How is ghc currently built on windows without something like cygwin?
From the source distribution, the only way to build ghc seems to be via configure and (GNU) make. So there must be some shell environment available.
Or am I missing something really crucial here? Ciao, Kili
Matthias Kilian wrote:
On Wed, Jan 30, 2008 at 08:13:01PM +1100, Manuel M T Chakravarty wrote:
[...]
This is due to a change of the configure stage that AFAIK was made to easy building on windows. Instead, of using shell commands/scripts (as GHC did previously) to obtain some configuration information (here the file path at which the top of the GHC build tree is located), the build system now uses small Haskell programs/scripts. This makes the build more portable ** if there is already a Haskell compiler on the system **.
But it just doesn't make sense at all. You need a good set of shell commands at all, since they're used by configure as well as in Makefiles. I really can't believe that simple stuff like this doesn't work on windos:
--- aclocal.m4.orig Mon Dec 10 19:11:31 2007 +++ aclocal.m4 Sun Jan 20 17:10:07 2008 @@ -1098,20 +1098,14 @@ AC_REQUIRE([AC_PROG_CC]) AC_DEFUN([FP_FIND_ROOT],[ AC_MSG_CHECKING(for path to top of build tree)
-dnl This would be -dnl make -C utils/pwd clean && make -C utils/pwd -dnl except we don't want to have to know what make is called. Sigh. -if test ! -f utils/pwd/pwd && test ! -f utils/pwd/pwd.exe; then - cd utils/pwd - rm -f *.o - rm -f *.hi - rm -f pwd - rm -f pwd.exe - $WithGhc -v0 --make pwd -o pwd - cd ../.. -fi - -hardtop=`utils/pwd/pwd forwardslash` +case $HostPlatform in +*cygwin32|*mingw32) + hardtop=`pwd | tr \\ /` + ;; +*) + hardtop=`pwd` + ;; +esac
if ! test -d "$hardtop"; then AC_MSG_ERROR([cannot determine current directory])
Things are complicated because - on Cygwin, pwd gives you /cygdrive/c/... - on MSYS, pwd gives you /c/... (remember we still support MSYS), and we want c:/... So we used to use cygpath on cygwin, and some horrible sed command on MSYS, IIRC. It was a mess, and frequently went wrong. Sure there are other ways to do it, but I think at the time it seemed simpler to write a Haskell program. In hindsight, probably a C program (compiled using mingw gcc) would be better for bootstrapping. A shell script would be problematic for the reasons above, I'm guessing. Cheers, Simon
Things are complicated because
- on Cygwin, pwd gives you /cygdrive/c/... - on MSYS, pwd gives you /c/...
(remember we still support MSYS), and we want c:/...
So we used to use cygpath on cygwin, and some horrible sed command on MSYS, IIRC. It was a mess, and frequently went wrong.
Sure there are other ways to do it, but I think at the time it seemed simpler to write a Haskell program. In hindsight, probably a C program (compiled using mingw gcc) would be better for bootstrapping. A shell script would be problematic for the reasons above, I'm guessing.
I understand that this is a difficult issues, but as it is booting from HC files and hence porting is simply broken. So, I wonder what the way forward is... Manuel
Manuel M T Chakravarty wrote:
Things are complicated because
- on Cygwin, pwd gives you /cygdrive/c/... - on MSYS, pwd gives you /c/...
(remember we still support MSYS), and we want c:/...
So we used to use cygpath on cygwin, and some horrible sed command on MSYS, IIRC. It was a mess, and frequently went wrong.
Sure there are other ways to do it, but I think at the time it seemed simpler to write a Haskell program. In hindsight, probably a C program (compiled using mingw gcc) would be better for bootstrapping. A shell script would be problematic for the reasons above, I'm guessing.
I understand that this is a difficult issues, but as it is booting from HC files and hence porting is simply broken. So, I wonder what the way forward is...
Porting is "simply broken" for various reasons at the moment - in particular the switch to Cabal for building the libraries broke it. We knew this at the time, but felt it was more important to make the switch now and fix bootstrapping later. For pwd, I imagine the best way forward is to use a C program, I don't think there are any deep issues there. Cheers, Simon
On Mon, Feb 04, 2008 at 12:18:01PM +0000, Simon Marlow wrote:
Porting is "simply broken" for various reasons at the moment - in particular the switch to Cabal for building the libraries broke it. We knew this at the time, but felt it was more important to make the switch now and fix bootstrapping later.
Be prepared: I'll send some comments and (probably silly) questions on cvs-ghc@ in a few minutes ;-)
For pwd, I imagine the best way forward is to use a C program, I don't think there are any deep issues there.
Unless some windows people have the time to step in and help make it work with simple shell stuff, some C helpers would be just fine. Ciao, Kili
On Wed, Jan 30, 2008 at 08:13:01PM +1100, Manuel M T Chakravarty wrote:
Philip K.F. Hölzenspies:
for f in `$(FIND) ghc-$(ProjectVersion)/compiler ghc-$(ProjectVersion)/rts -name "*.cmm" -follow -print` ""; do \ if test "x$$f" !=3D "x"; then \ echo `echo "$$f" | sed 's/cmm$$/hc/g' ` >> hc-files-to-go ; \ fi; \ done;
This adds some non-existing .hc files to the hc-files-to-go list that tar will not find, later on, causing an error. I just added a test to see whether the file exists. If it does not, I call make for that .hc file explicitly, which solves the problem for most files. The files that don't have a make target, I simply omitted from the hc-files-to-go. I haven't been able to test the severity of this, because of the second problem.
I am not sure what the problem is here. Simon? Ian?
I think there are some files that are meant to be missing; you can ignore the tar warnings/errors about them. I'm not sure what they are OTTOMH - possibly debug, prof, debug_prof etc variants of AutoApply. Thanks Ian
On Feb 3, 2008, at 5:35 PM, Ian Lynagh wrote:
I think there are some files that are meant to be missing; you can ignore the tar warnings/errors about them.
I'm not sure what they are OTTOMH - possibly debug, prof, debug_prof etc variants of AutoApply.
Dear Ian, All, They are indeed variants of AutoApply, but to simply ignore the tar error does not seem like a good idea. Tar terminates upon these errors and other stuff gets added to the hc-files-to-go later on in the Makefile. IMHO, if indeed these files are not required, the makefile should read: for f in `$(FIND) ghc-$(ProjectVersion)/compiler ghc-$(ProjectVersion)/ rts -name "*.cmm" -follow -print` ""; do\ fd="`echo $$f | sed 's/cmm$$/hc/g'`; \ if test "x$$f" != "x" -a -e "$$fd"; then \ echo $fd >> hc-files-to-go; \ fi; \ done; I don't know whether the -a argument for test is "common" test syntax, but it should give a logical. The -e option tests that "$$fd" exists and is a file. Regards, Philip
participants (6)
-
Ian Lynagh -
Manuel M T Chakravarty -
Matthias Kilian -
Philip Hölzenspies -
Philip K.F. Hölzenspies
-
Simon Marlow