March 2015 - ghc-devs

How to better parallelize GHC build.
by Karel Gardas 01 Apr '15

01 Apr '15

Folks, first of all, I remember someone already mentioned issue with decreased parallelism of the GHC build recently somewhere but I cann't find it now. Sorry, for that since otherwise I would use this thread if it was on this mailing list. Anyway, while working on SPARC NCG I'm using T2000 which provides 32 threads/8 core UltraSPARC T1 CPU. The property of this machine is that it's really slow on single-threaded work. To squeeze some perf from it man really needs to push 32 threads of work on it. Now, it really hurts my nerves to see it's lazy building/running just one or two ghc processes. To verify the fact I've created simple script to collect number of ghc processes over time and putting this to graph. The result is in the attached picture. The graph is result of running: gmake -j64 anyway, the average number of running ghc processes is 4.4 and the median value is 2. IMHO such low number not only hurts build times on something like CMT SPARC machine, but also on let say a cluster of ARM machines using NFS and also on common engineering workstations which provide these days (IMHO!) around 8-16 cores (and double the threads number). My naive idea(s) for fixing this issue is (I'm assuming no Haskell file imports unused imports here, but perhaps this may be also investigated): 1) provide explicit dependencies which guides make to build in more optimal way 2) hack GHC's make depend to kind of compute explicit dependencies from (1) in an optimal way automatically 3) someone already mentioned using shake for building ghc. I don't know shake but perhaps this is the right direction? 4) hack GHC to compile needed hi file directly in its memory if hi file is not (yet!) available (issue how to get compiling options right here). Also I don't know hi file semantics yet so bear with me on this. Is there anything else which may be done to fix that issue? Is someone already working on some of those? (I mean those reasonable from the list)? Thanks! Karel

5 7

Re: traverse_
by Herbert Valerio Riedel 31 Mar '15

31 Mar '15

On 2015-03-30 at 07:05:56 +0200, Fumiaki Kinoshita wrote: [...] > I found out that (<>) (in Data.Monoid) is missing, also. It would be nice > to reexamine Prelude to export things we want to export. Fwiw, (<>) was actually left-out as it wasn't required (it's just a an alias for `mappend`), *and* to keep our options open (or at least not make it more difficult) in terms of possible migration-plans available for the case we'd be moving 'Semigroup' to base/Prelude at some point in the future.

3 4

Re: traverse_
by Herbert Valerio Riedel 31 Mar '15

31 Mar '15

On 2015-03-31 at 06:27:32 +0200, Chris Wong wrote: [...] > While it would be useful to have it in the Prelude, traverse_ doesn't > fit either of the motivations above. So I think its omission is > intended here. That is not to say that it can't be proposed for inclusion... IOW, just turn the cautious "'traverse_' is not exported from Prelude [..] Is this intentional?"-question into an explicit/proper library proposal! Cheers, hvr

1 0

traverse_
by Kazu Yamamoto 30 Mar '15

30 Mar '15

Hi, I have just installed GHC 7.10.1. I'm a little bit surprised because 'traverse_' is not exported from Prelude. But 'traverse' is exported. Is this intentional? I expected that I can forget xxxM, xxxM_, xxxA, and xxxA_ functions with GHC 7.10.1. I want to use 'traverse_' instead of 'mapM_'. --Kazu

1 0

HP 2015.2.0.0 and GHC 7.10
by Randy Polen 30 Mar '15

30 Mar '15

I am helping Mark with the Haskell Platform, doing the Windows builds (32- and 64-bit). I want you to be aware of a problem I am encountering, and solicit suggestions and possible help. In building for HP 2015.2.0.0 on Windows 7, 64-bit (haven't gotten to 32-bit yet but likely the same problem will occur), I seem to be hitting the 32K limit for the length of arguments to a process, encountered while cabal is invoking haddock to build the docs for the OpenGLRaw package. For HP2014.2.0.0, the argument list was ~25K (from looking at my old build logs) but now is ~36K, which exceeds the maximum for CreateProcess (not a limit of the command-line, but of the OS call itself). Is there a way to build haddock docs for a single package but in multiple haddock invocations (maybe building a .haddock file for portions, then combining them, with the goal that the command line is kept short)? Seems this would also require a corresponding cabal change, as cabal is the invocator when this happens. Barring any existing mechanism, the typical solution to this problem on the Windows OS is (when possible, of course) to modify the program to accept a "response file" of command-line arguments. In this case, we could add an option to haddock to accept either a complete "response file" (i.e., allowing *all* options and arguments to come from a file) or just a file containing the files to process. Either of these changes to haddock are rather trivial to write (but adding another option implies more testing, documentation, other cases to handle, etc.). Since haddock ships with the ghc release, that's another wrinkle for this particular release. The other implication of such a solution is that cabal would need a change to utilize this change for it to be effective, checking haddock's version for support of this new haddock-flag, and either use it if the haddock version supports it, or do it optionally (which implies a new flag for cabal's haddock sub-command). This change to cabal is also rather trivial to implement (this is not to imply insensitivity to the incurred cost of each line of code, nor to the added burden of user-visible changes such as a command-line option). (Less desirable possibilities, mentioned only for completeness: skip the documentation for OpenGLRaw for this version of the Haskell Platform; split up the OpenGLRaw package itself in some way.) Other possible solutions and work-arounds? Thoughts on either using haddock in a different way (and the cabal change that would be required to break up the doc build into multiple steps for a single package)? Thoughts on the "response file" solution? Randy

3 4

Platform 2015.2.0.0 Alpha 2 for debian based systems
by Mark Lentczner 30 Mar '15

30 Mar '15

I've uploaded a build of *Haskell Platform 2015.2.0.0 Alpha 2* for generic debian linux systems. It includes GHC 7.10.2, using the generic linux bindist. This is a tarball, not a package. Installation instructions: cd / sudo tar xvf ...downloaded-tarfile... sudo /usr/local/haskell/ghc-7.10.1-x86-64/bin/activate-hs Notes: - Built on Ubuntu 14.04LTS - Needs the following packages installed: build-essential libbsd-dev libgmp-dev libtinfo-dev zlib1g-dev - The OpenGL packages need the OpenGL libs, which on a non-GUI system like a server can be gotten by installing: freeglut3-dev libgl1-mesa-dev libglu1-mesa-dev - Mark

1 1

OS X builds of GHC 7.8.4, 7.10.1-rc2, and 7.10.1-rc3
by Miëtek Bak 29 Mar '15

29 Mar '15

An OS X build of GHC 7.10.1-rc3 is now available, courtesy of Mark Lentczner. This is in addition to the GHC 7.8.4 and 7.10.1-rc2 builds Mark has made available on February 2. Thanks, Mark. GHC 7.8.4 for OS X can already be downloaded from haskell.org, even though it’s not mentioned on the GHC 7.8.4 webpage: https://downloads.haskell.org/~ghc/7.8.4/ghc-7.8.4-x86_64-apple-darwin.tar.… To avoid running up Mark’s bandwidth bill, I’m omitting the URL to his staging area. I’ve mirrored all three builds in Halcyon public storage: 7.8.4: https://halcyon.global.ssl.fastly.net/original/ghc-7.8.4-x86_64-apple-darwi… 7.10.1-rc2: https://halcyon.global.ssl.fastly.net/original/ghc-7.10.0.20150123-x86_64-a… 7.10.1-rc3: https://halcyon.global.ssl.fastly.net/original/ghc-7.10.0.20150316-x86_64-a… Binary builds of GHC, repackaged with documentation removed, are also available in Halcyon — on OS X 10.10, 10.9, 10.8, and a considerable number of Linux distributions. This includes GHC 7.10.1-rc3, 7.10.1-rc2, 7.8.4, 7.8.3, 7.8.2, 7.6.3, 7.6.1, 7.4.2, 7.2.2, and 7.0.4. Using Halcyon, installing GHC together with cabal-install is expected to take 20-30 seconds: https://halcyon.sh/tutorial/#install-ghc-and-cabal -- Miëtek https://mietek.io

1 1

Automating GHC build for Windows
by Roman Kuznetsov 29 Mar '15

29 Mar '15

Hello *, As I am still new in this, I will ask. Were there any attempt to automate GHC build process on Windows with some kind of CI engine, like Hudson, Jenkins, etc.? It seems to be rather helpful to publish build results if not after each commit, but at least once every day. WDYT? -- Sincerely yours, Roman Kuznetsov

7 16

Re: GHC 7.10.1 bindist for OS X + Haskell Platform 2015.2.0.0 Alpha 2
by Mark Lentczner 29 Mar '15

29 Mar '15

Verfied that the upload of the ghc bindist was cut short. New upload in progress! I've double checked and the build of Platform, and it's upload is fine.

1 1

Re: GHC 7.10.1 bindist for OS X + Haskell Platform 2015.2.0.0 Alpha 2
by Bob Ippolito 28 Mar '15

28 Mar '15

I'm having the same issue with that build. On Sat, Mar 28, 2015 at 3:42 PM, Alec <alec(a)deviant-logic.net> wrote: > Thanks Mark! But: > > % shasum -a 256 ghc-7.10.1-x86_64-apple-darwin.tar.bz2 > f7b112577966391988d16cb44061602f26bbe9a341aa6f4f1301d257895f1c65 > ghc-7.10.1-x86_64-apple-darwin.tar.bz2 > > Also, `tar xjf` didn't like it very much. Did your upload get cut off? > The tarball is only 77 Megs. > > -A > > On Sat, Mar 28, 2015 at 6:23 PM Mark Lentczner <mark.lentczner(a)gmail.com> > wrote: > >> I have built the 7.10.1 bindist for OS X. >> It is currently sitting on my server at: >> ghc-7.10.1-x86_64-apple-darwin.tar.bz2 >> <http://www.ozonehouse.com/mark/platform/ghc-7.10.1-x86_64-apple-darwin.tar.…> >> \O/ >> >> I have built an Alpha 2 of Haskell Platform 2015.2.0.0 for OS X >> <http://www.ozonehouse.com/mark/platform/Haskell%20Platform%202015.2.0.0%20a…>. >> It is based on the above bindist. >> I should have an ubuntu version of the Platform built by tomorrow. >> \O/ >> >> - Mark >> >> *sha256 sums:* >> c6eb8d573538eba27dfb2d9acf52a1beaaa512208261528b91e8d1775d133eee >> ghc-7.10.1-x86_64-apple-darwin.tar.bz2 >> 0cb351281fcf5ef34c54b455a3643d0ff078299a53b2c39d911dbda4aac2d233 Haskell >> Platform 2015.2.0.0 alpha 2 64bit.pkg >> >> >> _______________________________________________ >> Libraries mailing list >> Libraries(a)haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries >> > > _______________________________________________ > Libraries mailing list > Libraries(a)haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries > >

2 1