- ghc-devs - Haskell.org

Building GHC in a small amount of memory, in parts.
by Aaron Friel 19 Dec '13

19 Dec '13

I am trying to build GHC-HEAD and produce a reliable Dockerfile (see: docker.io) to allow others to use GHC-HEAD and also to make my own build process more reliable. I would like advice/assistance on two things to help this. For reference, I am using the stackbrew/ubuntu:13.10 image as my starting point, and bootstrapping GHC to HEAD using the current GHC (7.6.x) in Ubuntu's repositories. First issue: I am having issues with constraining the resource consumption during the build and have resorted to a few hacks such as replacing ld with gold (or rather, a script that filters out --hash-size arguments) and a custom build.mk file that uses -fasm for stage1. I would also like the final (stage 2?) result to be compiled with llvm, perhaps even without requiring gcc installed at all. How can I continue to push down the resource requirements of a build, and end up with a GHC that represents the bleeding edge? (That is what I want to be testing against, finding bugs for, etc.) Here is my build.mk: ################## BUILD.MK # Default to a very fast build for compilation and stage1: SRC_HC_OPTS = -H64m -O0 GhcStage1HcOpts = -O -fasm # But then use perf-llvm settings for remaining settings: GhcStage2HcOpts = -O2 -fllvm GhcHcOpts = -Rghc-timing GhcLibHcOpts = -O2 -fllvm -XGenerics GhcLibWays += p DYNAMIC_BY_DEFAULT = NO DYNAMIC_GHC_PROGRAMS = NO # Disable building all docs HADDOCK_DOCS = NO BUILD_DOCBOOK_HTML = NO BUILD_DOCBOOK_PS = NO BUILD_DOCBOOK_PDF = NO ################## BUILD.MK Second issue: These seem to bring memory usage down, but the monolithic ".\make" command is difficult to debug and very time consuming. When building a container image from a Dockerfile, the commands used are cached so that subsequent attempted builds use the intermediate results. So, even though my Dockerfile starts out with "apt-get update; apt-get upgrade -y", I don't have to do that every time I retry my build. Currently my Dockerfile pulls all the requisite packages from Ubuntu's repos, downloads and builds the latest cabal-install and builds alex, happy, hscolour and haddock, gets the ghc repo from GitHub, does "perl boot" ".\configure" and then I hit a wall with the monstrous amount of work that ".\make" does. I have tried splitting it up by doing ".\make stage=1", ".\make 1", and various combinations, but I hit dependency issues. Is there a way to break up the build into multiple commands?

1 0

eqExprX - dead code?
by Joachim Breitner 17 Dec '13

17 Dec '13

Hi, I came across some seemingly dead code, but I’d like to ask here before I clean it up. (Generally, cleaning up dead code is desirable, right?) The function eqExprX takes a parameter id_unfolding_fun which allows it to look through variable unfoldings. But the only use of this fuction is in eqExpr, which passes "const noUnfolding". Therefore, the two fancy cases in lines 1351-1362 never apply: http://git.haskell.org/ghc.git/blob/HEAD:/compiler/coreSyn/CoreUtils.lhs#l1… Some archeology: In Dec 2010, SPJ removed a use of eqExprX from the CSE module, which made use of that parameter (aae915d6). Thinking about it, it is probably safe to clean up the code. And if not, I’m not going to be offended by a "git revert", but just doing the change instead of waiting for permission seems to be more efficient to me. Greetings, Joachim -- Joachim “nomeata” Breitner mail(a)joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata(a)joachim-breitner.de • GPG-Key: 0x4743206C Debian Developer: nomeata(a)debian.org

1 0

arm-apple-darwin10-cabal install woes
by Schell Scivally 16 Dec '13

16 Dec '13

Hi all, I posted this earlier to the haskell-iphone list and heard it may be better off here. I'm having trouble installing primitive from hackage with arm-apple-darwin10-cabal. It seems a funky error is happening that is not getting written out or logged. Here is the output from my attempted install from hackage after a `rm -rf ~/.cabal` and `arm-apple-darwin10-cabal update`: http://lpaste.net/97092. Nothiing is listed in the log mentioned. So I tried installing from the git repo and got the same error. Then I tried to configure build and then install and the build phase spat out: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: archive library: dist-arm/build/libHSprimitive-0.5.2.0.a will be fat and ar(1) will not be able to operate on it In-place registering primitive-0.5.2.0... So it seems it's building but not linking? Maybe I shouldn't be developing with cabal? What's the normal procedure here? -- Schell Scivally http://blog.efnx.com http://github.com/schell http://twitter.com/schellsan

1 0

Using features from containers-0.5
by Joachim Breitner 16 Dec '13

16 Dec '13

Hi, especially to Austin with his Release Manager hat on, in my nested-cpr branch I have a few nice patches that clean up, refactor and especially Note’ify the demand analyzer. I’d like to see both my branch to be shorter, and to see those improvements in the mainline independent of whether nested-cpr will turn out to be useful, so I invested some time to rebase my branch and move all these patches below those that actually change the code. The result is the contents of the wip/nested-cpr branch up to and including the patch “Make types of bothDmdType more precise”, which I find especially useful (currently c26d31, but I tend to rebase while I add more polish or remove typos). I am confident that this is a pure refactoring (validate passes without changes to the testsuite, nofib shows identical Allocation numbers), and I’d like to push this to master. But one of the patches, “Clarify the default demand on demand environments” (currently 2b6a6a), uses a function from containers (IntMap.mergeWithKey) which was only added in containers-0.5, which ships with GHC 7.6, but not with 7.4. But especially that change is nice, as removes the confusing modifyEnv function and makes the code (IMHO) easier to understand. What is the policy here? Is it ok to expect the user to upgrade the containers package of the bootstrap compiler? Or can we start building containers before stage-1, so that we can expect the new code (but when I tried it seems that this needs some adjustments to the build system)? Or should I add an alternative definition via CPP in compiler/utils/UniqFM.lhs? Or is it not worth it and I should wait until 7.9 (which would then drop the requirement to be bootstrappable with 7.4, right?) Thanks, Joachim -- Joachim “nomeata” Breitner mail(a)joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata(a)joachim-breitner.de • GPG-Key: 0x4743206C Debian Developer: nomeata(a)debian.org

2 2

Ways issues with GhcDynamic=YES
by Ben Gamari 15 Dec '13

15 Dec '13

I've spent much of the day trying to determine the root cause of, ... "inplace/bin/ghc-cabal" register libraries/ghc-prim dist-install "/opt/exp/ghc/root-ghc-llvm-dyn-7.8/lib/ghc-7.7.20131214/bin/ghc" "/opt/exp/ghc/root-ghc-llvm-dyn-7.8/lib/ghc-7.7.20131214/bin/ghc-pkg" "/opt/exp/ghc/root-ghc-llvm-dyn-7.8/lib/ghc-7.7.20131214" '' '/opt/exp/ghc/root-ghc-llvm-dyn-7.8' '/opt/exp/ghc/root-ghc-llvm-dyn-7.8/lib/ghc-7.7.20131214' '/opt/exp/ghc/root-ghc-llvm-dyn-7.8/share/doc/ghc/html/libraries' NO ghc-cabal: Bad interface file: dist-install/build/GHC/CString.hi mismatched interface file ways (wanted "dyn", got "") I'm attempting to produce a build to validate dynamic linking with LLVM so build.mk looks something like, DYNAMIC_BY_DEFAULT = YES BuildFlavour = quick-llvm Looking at the interface files produced for ghc-prim, it seems that ghc-cabal is looking for the wrong extension, $ inplace/bin/ghc-stage2 --show-iface libraries/ghc-prim/dist-install/build/GHC/CString.hi | grep --after-context=1 Way Way: Wanted [d, y, n], got [] $ inplace/bin/ghc-stage2 --show-iface libraries/ghc-prim/dist-install/build/GHC/CString.dyn_hi | grep --after-context=1 Way Way: Wanted [d, y, n], got [d, y, n] Any ideas what might be going on here? How is `ghc-cabal` supposed to know which interface file to use? Cheers, - Ben

1 0

Cross-Compiler: Windows-to-iOS
by Alex Carter 13 Dec '13

13 Dec '13

Hello, ghc-devs! This is continuation of a conversation from the iphone list, but since it seems to be more of a general cross-compilation issue, I'll toss it up for you guys. (See the conversation history below if you're curious.) The gist is that I'm trying to build a Windows-to-iOS cross-compiler, and since I imagine this isn't something done very often, I've hit a few bumps along the way. First up, gcc (4.5.2, inplace/mingw/bin) really doesn't seem to like rts/Task.h:283, complaining "error: aggregate value used where an integer was expected." For this configuration, presumably OSThreadId is pthread_t and TaskId is uint64, so I'm not sure why it doesn't like the casts. I worked around with return *( (TaskId*) ( (size_t*) &taskID ) ); Last night I was stuck on compiling some automatically generated headers that come out of the deriveConstants utility, and as best I can gather, it generates a header that is (marginally) dependent upon the platform on which it's currently running by using System.Info.os, but the compilation that uses the header seems to be expecting it to work for the target (runtime, maybe?). I added an "os" switch to deriveConstants rather than using System.Info.os and passed the target OS through in includes/ghc.mk where it's invoked. So that seems to fix my issues with StgAsyncIOResults (Windows-only, right?) but I have no idea if these hacks make any sense in the big picture, or if the compiler will even work once I finish the build... Presently I'm stuck on something I expected to be rather straightforward: "Warning: Couldn't figure out LLVM version! Make sure you have installed LLVM." The iOS toolchain I'm using is in my PATH, and both "llc -version" and "opt -version" work as expected in my shell. I'm curious if it's because of the odd version number: $ llc -version LLVM (http://llvm.org/) LLVM version 3.3svn ... Is ./configure perhaps getting caught up on the "svn" and treating this as a failure of version detection? I haven't been able to find a way to manually override the configuration and specify the LLVM paths. Any help is appreciated so I can get back to writing Haskell instead of build debugging. ;) Thanks, Alex On Wed, Dec 11, 2013 at 12:27 AM, Simon Marlow <marlowsd(a)gmail.com> wrote: > Yes, this is correct. ghc_boot_platform describes the platforms for the > stage 1 compiler (assuming you're looking at the one in compiler/stage1). > ghcplatform.h describes the platforms in use for compiled code, so the host > in that case is ios. In fact ghcplatform.h shouldn't have a target platform > at all - the only reason it does is for "backwards compatibility", though I > don't remember exactly what goes wrong if we remove these. > > Cheers, > Simon > > > On 11/12/2013 00:08, Alex Carter wrote: >> >> I'm on an interesting journey building a GHC cross-compiler on Windows >> to target iOS, using a third-party iOS toolchain for Windows >> (http://www.pmbaty.com/iosbuildenv/ for the curious), and I'm running >> into an unusual issue. Since this is a cross-compiler, this should be >> all stage 1, so in ghc_boot_platform.h I see what I expect: >> >> ... >> #define BUILD_OS "mingw32" >> #define HOST_OS "mingw32" >> #define TARGET_OS "ios" >> ... >> >> but ghcplatform.h shows: >> >> ... >> #define BUILD_OS "mingw32" >> #define HOST_OS "ios" >> ... >> #define TARGET_OS "ios" >> ... >> >> which I understand to mean that whatever's happening there is set up >> for the stage 2 compiler (unless there's some good magic happening >> here with the deriveConstants utility?). I discovered this through a >> pragma added at the error site: >> >> ... >> rts/Task.h:280:9: note: #pragma message: HOST_OS=ios >> rts/Task.h:287:5: error: aggregate value used where an integer was >> expected >> ... >> >> Anybody know what's going on here? Is it right (for some reason beyond >> my current comprehension) for ghcplatform.h to have the #defines it >> does? Is HOST_OS=ios correct, but rts/Task.h:280 should also check >> defined(ios_HOST_OS)? >> >> Thanks for the help, >> Alex >> >> >> >> _______________________________________________ >> iPhone mailing list >> iPhone(a)haskell.org >> http://www.haskell.org/mailman/listinfo/iphone >> >

1 0

HeapStackCheck.cmm
by Alexander Pakhomov 11 Dec '13

11 Dec '13

Hi. Trying to compile ghc HEAD on OpenBSD I got an error: rts/HeapStackCheck.cmm:97:18: parse error on input `[' The code is following: 96 if (HpAlloc <= BLOCK_SIZE 97 && bdescr_link(CurrentNursery) != NULL) { which preprocesses to: if (HpAlloc <= (1<<12) && b0[CurrentNursery+15] != (0::bits64)) { This is the first occurrence of b0 in preprocessed file. I believe that it is due to undefined b0. Command that fails is: "inplace/bin/ghc-stage1" -static -H32m -O -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header -Iincludes/dist-ghcconstants/header -Irts -Irts/dist/build -DCOMPILING_RTS -package-name rts -dcmm-lint -i -irts -irts/dist/build -irts/dist/build/autogen -Irts/dist/build -Irts/dist/build/autogen -O2 -c rts/HeapStackCheck.cmm -o rts/dist/build/HeapStackCheck.o To preprocess do the following: cpp -O -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header -Iincludes/dist-ghcconstants/header -Irts -Irts/dist/build -DCOMPILING_RTS -Irts/dist/build -Irts/dist/build/autogen rts/HeapStackCheck.cmm | less Can you please find source of b0 in preprocessed file?

3 5

Please help complete `.mailmap`
by Herbert Valerio Riedel 11 Dec '13

11 Dec '13

Hello *, I've been trying to (re)construct a .mailmap file[1] in ghc.git which allows `git shortlog` and `git log --use-mailmap` (and possibly other tools/commands) to fixup bogus author-ids when displaying the Git history, in the hopes to provide a more complete history of GHC and all contributors involved. I've been able to infer mappings for most users, but I'm stuck on a couple of older account usernames dating back a decade ago for which I have no idea what real names to associate with. These are specifically: $ git shortlog -es | grep unknown 16 apt <unknown> 1 cryder <unknown> 13 desrt <unknown> 8 dnt <unknown> 1 geoffw <unknown> 12 gla <unknown> 3 josefs <unknown> 5 krc <unknown> 1 kw <unknown> 5 kw217 <unknown> 1 mbs <unknown> 1 moran <unknown> 3 njn <unknown> 8 ralf <unknown> 11 rje <unknown> 1 sseefried <unknown> 1 tgedell <unknown> 1 uid245 <unknown> 1 umuta <unknown> 1 y.zhuang5(a)lse.ac.uk <unknown> If you happen to know who these past contributors are, please either tell me or update [1] accordingly. Moreover, if you stumble over any wrong entry in [1], feel free to correct it. Thanks, hvr [1] http://git.haskell.org/ghc.git/blob/HEAD:/.mailmap PS: If anyone's curious, the top-5 committers in ghc.git are currently: $ git shortlog -sn | head -n5 8513 Simon Marlow 5368 Simon Peyton Jones 4629 Ian Lynagh 2633 Sigbjorn Finne 941 Julian Seward

4 3

nofib’s cacheprof’s Allocations not deterministic
by Joachim Breitner 10 Dec '13

10 Dec '13

Hi, with most of nofib, one expects at least the Allocation numbers to be reliably and deterministic. This is not the case with cacheprof, they seem to wibble around by ~0.2%, which is strange. Has anyone else noticed this yes, and if so, tried to find out the cause? Is it expected and desired, or a bug somewhere? Greetings, Joachim -- Joachim “nomeata” Breitner mail(a)joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nomeata(a)joachim-breitner.de • GPG-Key: 0x4743206C Debian Developer: nomeata(a)debian.org

3 2

benchmarks
by Alexander Pakhomov 10 Dec '13

10 Dec '13

Hi. Do you do performance testing? I would be interested in statistics by release. I worked at optimizing compiler at my previous job and we had quite huge amount of performance tests for each commit. When something slowed down we fixed it. It is strange but most of times we found and fixed slowdown.

3 2