- Glasgow-haskell-users

RE: Preparing a Registered hc-file-bundle (Was BootstrappingwithHCfiles)
by Simon Marlow 16 Dec '05

16 Dec '05

On 15 December 2005 16:19, Andrew Walrond wrote: > Hi Simon, > > On Wednesday 14 December 2005 09:49, Simon Marlow wrote: >> Actually, looking at ghc/Makefile, it is possible that ghc/lib isn't >> getting built early enough. Try changing: >> >> SUBDIRS = includes rts docs compiler lib utils driver >> >> to >> >> SUBDIRS = includes rts lib docs compiler utils driver >> > > Did that. Now I get to here: > > ------------------------------------------------------------------------ > ===fptools== Recursively making `all' in compat ... > PWD = /home/andrew/test/ghc/bs/target/ghc-6.4.1/ghc/lib > ------------------------------------------------------------------------ > ------------------------------------------------------------------------ > ==fptools== make all - --no-print-directory -r; > in /home/andrew/test/ghc/bs/target/ghc-6.4.1/ghc/lib/compat > ------------------------------------------------------------------------ > make[2]: *** No rule to make target `Compat/Directory.o', needed by > `ghccompat.o'. Stop. > make[1]: *** [all] Error 1 > make: *** [all] Error 1 > make: Leaving directory > `/home/andrew/test/ghc/bs/target/ghc-6.4.1/ghc' > > Any clues how to proceed? I have no idea what has gone wrong here. We ought to be able to make Compat/Directory.o. However, we don't need ghccompat.o, so you could try adding: GhcWithInterpter=NO to ghc/lib/compat/Makefile, right after the NO_INSTALL_LIBRARY setting. If that doesn't work, please run 'make -p -d' in ghc/lib/compat and send me the output. Cheers, Simon

2 1

RE: utf8 strings: memory optimization and case-ignoring comparision
by Simon Marlow 16 Dec '05

16 Dec '05

On 14 December 2005 20:35, Bulat Ziganshin wrote: > i use utf8-packed strings in my program and have to ask 2 questions > about them: > > 1. i need function to do case-ignoring comparision of such strings. > stricmp is not appropriate because it don't know about utf8. can be > the existing Unicode support in Data.Char used for these or can the > appropriate support will be added? you should be able to use toUpper/toLower from Data.Char in GHC 6.4.1. > 2. what is the most memory-efficient representaion for such strings? > now i use John Meacham's library > (http://repetae.net/john/repos/jhc/PackedString.hs) which declares: > > newtype PackedString = PS (UArray Int Word8) > > but this uses two Ints just to hold index bounds: > > data UArray i e = UArray !i !i ByteArray# I don't know why an extra 8/16 bytes per string is that worrying - if you have so many small strings perhaps you should be sharing them via a hash table? > i want to use just memory ptr and put NUL at the end of array (my > strings never contain NUL chars). but what type i must use for this > ptr? ByteArray/ByteArray#, ForeignPtr, StablePtr, Ptr?? and which > function > i must use to quickly allocate memory i need? my packed strings will > be only unpacked and passed to "unsafe" C functions: stricmp, strcpy, > strcat; i plan to not use any other operations ForeignPtr and mallocForeignPtr are the way to go these days. In GHC 6.6 these will be much faster than before. Cheers, Simon

2 1

RE: Preparing a Registered hc-file-bundle (Was Bootstrapping withHCfiles)
by Simon Marlow 15 Dec '05

15 Dec '05

On 14 December 2005 09:46, Simon Marlow wrote: > On 13 December 2005 18:58, Andrew Walrond wrote: > >> Well, I gave it a blast anyway. All went well creating the hc files >> until the last command: >> >>> $ make hc-file-bundle Project=Ghc >> >> at which point it failed with >> >> tar czf ghc-6.4.1-x86_64-unknown-linux-hc.tar.gz `cat hc-files-to-go` >> tar: ghc-6.4.1/ghc/rts/AutoApply_thr.hc: Cannot stat: No such file >> or directory tar: ghc-6.4.1/ghc/rts/AutoApply_thr_p.hc: Cannot stat: >> No such file or directory tar: ghc-6.4.1/ghc/rts/AutoApply_debug.hc: >> Cannot stat: No such file or directory tar: Error exit delayed from >> previous errors >> make: *** [hc-file-bundle] Error 2 > > Yes, these are harmless. They are caused by the fact that > AutoApply.cmm is auto-generated in several different ways > (AutoApply_thr.cmm, AutoApply_debug.cmm etc.), and the .hc files for > these are named AutoApply_thr.thr_hc, AutoAPply_debug.debug_hc, and > so on. The Makefile code for hc-bundle doesn't know about this > special case. > >> Googling suggests this might be harmless, so I tried bootstrapping on >> the target machine using the created tarball, but that failed with >> >> /bin/ld: cannot find -lghccompat >> collect2: ld returned 1 exit status >> make[1]: *** [stage1/ghc-6.4.1] Error 1 >> make: *** [all] Error 1 >> make: Leaving directory >> `/home/andrew/test/ghc/bs/target/ghc-6.4.1/ghc' > > Please send us a full log of the build. libghccompat.a should be in > ghc/lib/compat, and it should be built quite early on. Actually, looking at ghc/Makefile, it is possible that ghc/lib isn't getting built early enough. Try changing: SUBDIRS = includes rts docs compiler lib utils driver to SUBDIRS = includes rts lib docs compiler utils driver Cheers, Simon

2 1

utf8 strings: memory optimization and case-ignoring comparision
by Bulat Ziganshin 14 Dec '05

14 Dec '05

Hello glasgow-haskell-users, i use utf8-packed strings in my program and have to ask 2 questions about them: 1. i need function to do case-ignoring comparision of such strings. stricmp is not appropriate because it don't know about utf8. can be the existing Unicode support in Data.Char used for these or can the appropriate support will be added? 2. what is the most memory-efficient representaion for such strings? now i use John Meacham's library (http://repetae.net/john/repos/jhc/PackedString.hs) which declares: newtype PackedString = PS (UArray Int Word8) but this uses two Ints just to hold index bounds: data UArray i e = UArray !i !i ByteArray# i want to use just memory ptr and put NUL at the end of array (my strings never contain NUL chars). but what type i must use for this ptr? ByteArray/ByteArray#, ForeignPtr, StablePtr, Ptr?? and which function i must use to quickly allocate memory i need? my packed strings will be only unpacked and passed to "unsafe" C functions: stricmp, strcpy, strcat; i plan to not use any other operations -- Best regards, Bulat mailto:bulatz@HotPOP.com

1 0

RE: Preparing a Registered hc-file-bundle (Was Bootstrapping withHC files)
by Simon Marlow 14 Dec '05

14 Dec '05

On 13 December 2005 18:58, Andrew Walrond wrote: > Well, I gave it a blast anyway. All went well creating the hc files > until the last command: > >> $ make hc-file-bundle Project=Ghc > > at which point it failed with > > tar czf ghc-6.4.1-x86_64-unknown-linux-hc.tar.gz `cat hc-files-to-go` > tar: ghc-6.4.1/ghc/rts/AutoApply_thr.hc: Cannot stat: No such file or > directory > tar: ghc-6.4.1/ghc/rts/AutoApply_thr_p.hc: Cannot stat: No such file > or directory > tar: ghc-6.4.1/ghc/rts/AutoApply_debug.hc: Cannot stat: No such file > or directory > tar: Error exit delayed from previous errors > make: *** [hc-file-bundle] Error 2 Yes, these are harmless. They are caused by the fact that AutoApply.cmm is auto-generated in several different ways (AutoApply_thr.cmm, AutoApply_debug.cmm etc.), and the .hc files for these are named AutoApply_thr.thr_hc, AutoAPply_debug.debug_hc, and so on. The Makefile code for hc-bundle doesn't know about this special case. > Googling suggests this might be harmless, so I tried bootstrapping on > the target machine using the created tarball, but that failed with > > /bin/ld: cannot find -lghccompat > collect2: ld returned 1 exit status > make[1]: *** [stage1/ghc-6.4.1] Error 1 > make: *** [all] Error 1 > make: Leaving directory > `/home/andrew/test/ghc/bs/target/ghc-6.4.1/ghc' Please send us a full log of the build. libghccompat.a should be in ghc/lib/compat, and it should be built quite early on. Cheers, Simon

1 0

RE: Preparing a Registered hc-file-bundle (Was Bootstrapping with HCfiles)
by Simon Marlow 14 Dec '05

14 Dec '05

On 13 December 2005 11:05, Andrew Walrond wrote: > On the HOST > ----------------- > > Unpack src tree, and cd into it. > > Configure with necessary options > $ ./configure --prefix=... > > Section 10.2.1'Cross-compiling to produce an unregisterised GHC' in > the build manual suggests creating an mk/build.mk with following > contents: GhcUnregisterised = YES > GhcLibHcOpts = -O -fvia-C -keep-hc-files > GhcRtsHcOpts = -keep-hc-files > GhcLibWays = > SplitObjs = NO > GhcWithNativeCodeGen = NO > GhcWithInterpreter = NO > GhcStage1HcOpts = -O > GhcStage2HcOpts = -O -fvia-C -keep-hc-files > SRC_HC_OPTS += -H32m > GhcBootLibs = YES > Reading between the lines somewhat, for a full registered version we > might need something like > GhcUnregisterised = NO > GhcLibHcOpts = -O -fvia-C -keep-hc-files > GhcRtsHcOpts = -keep-hc-files > GhcLibWays = > SplitObjs = NO > GhcWithNativeCodeGen = YES > GhcWithInterpreter = YES > GhcStage1HcOpts = -O > GhcStage2HcOpts = -O -fvia-C -keep-hc-files > SRC_HC_OPTS += -H32m > GhcBootLibs = YES > Advice on this bit gratefully received! This should be fine, except that you can leave out some of it - you probably want the default settings for GhcWithNativeCodeGen and GhcWithInterpreter, and the GhcUnregisterised should default to NO. > The manual next suggests copying some target generated include files > into the host ghc source, but since they are the same I guess we can > ignore that bit. > > Now building exactly as described in the manual: > > $ pushd glafp-utils && make boot && make && popd > $ pushd ghc && make boot && make && popd > > Manual suggests the build might fail in RTS, but probably not for our > registered version ? building the RTS should work, yes. > $ pushd libraries && make boot && make && popd > $ pushd ghc/compiler && make boot stage=2 && make stage=2 && popd > > $ pushd ghc/lib/compat > $ make clean > $ rm .depend > $ make boot UseStage1=YES > $ make -k UseStage1=YES EXTRA_HC_OPTS='-O -fvia-C -keep-hc-files' > $ popd > $ pushd ghc/utils > $ make clean > $ make -k UseStage1=YES EXTRA_HC_OPTS='-O -fvia-C -keep-hc-files' > $ popd > $ make hc-file-bundle Project=Ghc > > That should provide us with our hc-file-bundle, *-hc.tar.gz > > On the TARGET > -------------------- > > Unpack src tree, unpack the hc file bundle on top of it. > Cd into the src tree. > > Now it should be a simple matter of > $ distrib/hc-build --prefix=dir > $ make install Looks good! Cheers, Simon

1 0

Preparing a Registered hc-file-bundle (Was Bootstrapping with HC files)
by Andrew Walrond 13 Dec '05

13 Dec '05

Ok, so now we know its possible! Now the question is how to create and use a _registered_ hc file bundle. The Build manual describes the process of cross compiling an _unregistered_ hc file bundle, but has nothing to say about creating the simpler registered hc file bundle for the purpose of bootstrapping GHC on an equivalent supported platform but without a preinstalled ghc. So, reading between the lines a little, here is what I propose trying. Please take a look, and give me your thoughts if you think I am on the wrong track! HOST: Has ghc, happy, alex installed. TARGET: Same as host, but without ghc, happy alex. On the HOST ----------------- Unpack src tree, and cd into it. Configure with necessary options $ ./configure --prefix=... Section 10.2.1'Cross-compiling to produce an unregisterised GHC' in the build manual suggests creating an mk/build.mk with following contents: GhcUnregisterised = YES GhcLibHcOpts = -O -fvia-C -keep-hc-files GhcRtsHcOpts = -keep-hc-files GhcLibWays = SplitObjs = NO GhcWithNativeCodeGen = NO GhcWithInterpreter = NO GhcStage1HcOpts = -O GhcStage2HcOpts = -O -fvia-C -keep-hc-files SRC_HC_OPTS += -H32m GhcBootLibs = YES Reading between the lines somewhat, for a full registered version we might need something like GhcUnregisterised = NO GhcLibHcOpts = -O -fvia-C -keep-hc-files GhcRtsHcOpts = -keep-hc-files GhcLibWays = SplitObjs = NO GhcWithNativeCodeGen = YES GhcWithInterpreter = YES GhcStage1HcOpts = -O GhcStage2HcOpts = -O -fvia-C -keep-hc-files SRC_HC_OPTS += -H32m GhcBootLibs = YES Advice on this bit gratefully received! The manual next suggests copying some target generated include files into the host ghc source, but since they are the same I guess we can ignore that bit. Now building exactly as described in the manual: $ pushd glafp-utils && make boot && make && popd $ pushd ghc && make boot && make && popd Manual suggests the build might fail in RTS, but probably not for our registered version ? $ pushd libraries && make boot && make && popd $ pushd ghc/compiler && make boot stage=2 && make stage=2 && popd $ pushd ghc/lib/compat $ make clean $ rm .depend $ make boot UseStage1=YES $ make -k UseStage1=YES EXTRA_HC_OPTS='-O -fvia-C -keep-hc-files' $ popd $ pushd ghc/utils $ make clean $ make -k UseStage1=YES EXTRA_HC_OPTS='-O -fvia-C -keep-hc-files' $ popd $ make hc-file-bundle Project=Ghc That should provide us with our hc-file-bundle, *-hc.tar.gz On the TARGET -------------------- Unpack src tree, unpack the hc file bundle on top of it. Cd into the src tree. Now it should be a simple matter of $ distrib/hc-build --prefix=dir $ make install Any comments/suggestions before I give it a try? Thanks for all help so far, and in anticipation of more! Andrew Walrond

1 1

RE: GHC wish list.
by Simon Marlow 13 Dec '05

13 Dec '05

Hi Lemmih, Might be a good idea to add these as feature requests: http://hackage.haskell.org/trac/ghc/newticket?type=feature+request Cheers, Simon On 13 December 2005 15:17, Lemmih wrote: > I've been using the GHC library in hIDE and HASP for some time now and > there are a few things I'd love to see implemented: > > * Changeable lexer/parser (just like DynFlags.log_action) > Not very important but it would be convenient in HASP. > * Cabalization of the library. > Not having a profiling version of the library and not being able to > rebuild it easily is a bit limiting. > * GHCi compatibility. > hIDE is currently using ghc-api[1] because the real library exports > symbols that clashes with GHCi.

1 0

GHC wish list.
by Lemmih 13 Dec '05

13 Dec '05

Hiya, I've been using the GHC library in hIDE and HASP for some time now and there are a few things I'd love to see implemented: * Changeable lexer/parser (just like DynFlags.log_action) Not very important but it would be convenient in HASP. * Cabalization of the library. Not having a profiling version of the library and not being able to rebuild it easily is a bit limiting. * GHCi compatibility. hIDE is currently using ghc-api[1] because the real library exports symbols that clashes with GHCi. -- Friendly, Lemmih [1] ghc-api: http://scannedinavian.com/~lemmih/ghc-api

1 0

RE: Bootstrapping with HC files
by Simon Marlow 13 Dec '05

13 Dec '05

On 12 December 2005 18:21, Andrew Walrond wrote: > 3) Why don't you guys supply hc tarballs for bootstrapping on common > platforms? They would seem to be rather more versatile than the usual > rpms and binary tarballs. A couple of reasons, neither of which is very convincing: 1. the HC files are sensitive to platform differences, so I'm not sure that HC files generated for one flavour of Linux would necessarily work on other flavours, for example. 2. building from HC files isn't well supported, and tends to be rather fragile, so we don't like to encourage it. The reason it isn't well supported is just lack of time. Cheers, Simon

1 0