- ghc-devs - Haskell.org

[haskell.org Google Summer of Code] Call for Mentors
by Edward Kmett 07 Apr '15

07 Apr '15

We have had a rather large pool of potential students apply for this year's Google Summer of Code, but, ultimately, Google won't let us ask for a slot unless we have a potential mentor assigned in advance. On top of that, one thing we try to do with each project is wherever possible, assign both a primary and a backup mentor, so the available mentoring pool is drawn a little thin. Many hands make for light work, though: If you've mentored or thought about mentoring in years past, I'd encourage you to sign up on google-melange for the Google Summer of Code at: https://www.google-melange.com/gsoc/homepage/google/gsoc2015 and request a connection to haskell.org as a Mentor. Once you've done this you can help us vote on proposals, and should something seem appropriate to you, you can flag yourself as available as a potential mentor or backup mentor for one (or more) of the projects. We have a couple of weeks left to rate proposals and request slots, but it'd be good to make as much progress as we can this week. If you have any questions, feel free to reach out to me, or to Shachaf Ben-Kiki or Gershom Bazerman who have been helping out with organizational issues this year. We also have a #haskell-gsoc channel on irc.freenode.net if you have questions about what is involved. Thank you for your time and consideration, -Edward Kmett

1 0

Use of GHC in Open Embedded Environments
by dpquigl 06 Apr '15

06 Apr '15

Hello Everyone, I sent a couple of private messages to Simon and he suggested I post to the list instead to get more eyes on the problem. I have been working for the better part of 3 days on trying to get GHC to build within an Open Embedded environment. Open Embedded is used to set up embedded build environments that are completely isolated from the build platform. The problem I'm encountering is due to the way GHC builds its requiring the build and host platforms to be the same. I want to create a GHC build where the build and host are different so I can guarantee the isolation of the Haskell used to build our programs and libraries on target. I looked extensively through the cross compilation wiki page and even tried specifying paths into configure script to specify the tools from within my toolchain to build with and I am still running into problems. It seems that even if the top level makefile gets configured properly the build environment for the modules created with cabal under libraries do not retain any of that configuration information. The reason this is important is that I'm working on OpenXT which is a recently Open Sources version of a product Citrix use to have called Xen Client XT. The idea of OpenXT is that we use Xen to decompose what is traditionally a monolithic system into several VMs which serve specific purposes. For example we have the user interface for the platform which manages things like sound levels and bringing up and shutting down VMs into its own domain called the UI domain. We also separate out networking into its own domain as well where the network card is mapped exclusively into that domain and it manages creating and running virtual network adapters for other VMs on the platform. We use haskell in two different ways in this system. We personally want to use OpenXT as a research platform and we have special purpose mini domains for doing things like measurement and attestation and we have components of these written in Haskell to do formal verification of the domains. What OpenXT is using Haskell for directly is as part of its management engine for the platform. We have a haskell and ocaml(just for glue) based versions of metadata storage and platform management APIs. Now when we build the platform we want to remove the dependency on the host platform GHC version. We try to do this by building what would essentially be a stage 1 compiler which will then be used to build the runtime and tools used in the final platform VMs. The issue is that the GHC build does not recognize the use case of host and build machines being different. They expect host and build to be the same and target to be different. Because of this even if we specify each component individually on the configure line for the base GHC build when the build gets to the point of building the libraries it seems to have this information completely vanish. I think this is because it is using cabal to build the libraries and cabal isn't taking into account that GHC is built for a second platform and we want to build those libraries for that same platform. Dave

2 1

DPH tests
by Konstantine Rybnikov 06 Apr '15

06 Apr '15

Hi! There’s «Additional Packages» section [0] which lists packages that need to be installed in order to run full testsuite. But I couldn’t install dph-* ones, first errors I was getting were regular cabal-constraint ones. I wrote an author Ben Lippmeier, but he replied that dph-* packages were obsolete and not maintained anymore. I grepped and saw that all tests are under testsuite/dph folder. So: does GHC still run them at all? Should we edit wiki? Or edit test-suite? Thanks! [0]: https://ghc.haskell.org/trac/ghc/wiki/Building/RunningTests/Running#Additio…

4 3

DynFlags.hs
by Kazu Yamamoto 06 Apr '15

06 Apr '15

Hi, I cannot build GHC 7.10.1 on CentOS 6.5 whose memory is just 1G. This is because compiler/main/DynFlags.hs is huge. It would be nice if this file could be split. --Kazu

3 2

GHC 7.10.1 html docs all flat?
by Randy Polen 02 Apr '15

02 Apr '15

The HP build expects that the html docs for the ghc-bundled packages will be in heirarchical directories, based on the package name (e.g., GHC.Magic docs in libraries/ghc-prim-0.3.1.0/GHC-Magic.html) which is how it has been for many ghc releases. In 7.10.1, I thought it was an error but it seems the haskell.org docs reflect this new, flat way (e.g,. GHC.Magic docs in libraries/GHC-Magic.html). Just want to make sure this is what is expected, and then change the HP build code accordingly. Follow up question: should the HP-bundled docs do the same thing? Is the flat model the way to go for all the docs? Won't this cause collisions for updated packages? Randy

3 2

Increased memory usage with GHC 7.10.1
by Jan Stolarek 02 Apr '15

02 Apr '15

Forall hi, I just uprgaded both of my machines to use GHC 7.10.1. I keep sandboxed installations of GHC and this means I had to rebuild Agda and Idris because the binaries built with GHC 7.8.4 were stored inside deactivated 7.8.4 sandbox. Sadly, I had problems building both Agda and Idris due to GHC taking up all of available memory. With Idris the problematic module was Idris.ElabTerm (~2900LOC). The interesting part of the story is that when I do a clean build of Idris GHC consumes all of memory when compiling that module and I have to kill the build. But when I restart the build after killing GHC the module is compiled using a reasonable amount of memory and within reasonable time. With Agda the problematic module is Agda.TypeChecking.Serialise (~2000LOC). The trick with killing the build and restarting it didn't work in this case. I had to compile Agda with GHC 7.8.4 (which works without problems though the mentioned module still requires a lot of memory) and alter my setup so that Agda binary is not stored inside GHC sandbox. I wonder if any of you came across similar issues with GHC 7.10.1? Do we have any performance data that allows to compare memory usage and performance of GHC 7.10.1 with previous stable releases? All of the above happened on 64bit Debian Wheezy with 2GB of RAM. Janek --- Politechnika Łódzka Lodz University of Technology Treść tej wiadomości zawiera informacje przeznaczone tylko dla adresata. Jeżeli nie jesteście Państwo jej adresatem, bądź otrzymaliście ją przez pomyłkę prosimy o powiadomienie o tym nadawcy oraz trwałe jej usunięcie. This email contains information intended solely for the use of the individual to whom it is addressed. If you are not the intended recipient or if you have received this message in error, please notify the sender and delete it from your system.

5 7

Typeable
by Simon Peyton Jones 02 Apr '15

02 Apr '15

Iavor, Richard, and others We always thought it was the Right Thing to generate the runtime representation for a TyCon (used in Typeable type representations) at the definition site of the type, not "on the fly". I thought I'd do this. But it was a LOT harder than I expected. I have spent a ridiculous amount of time on it over the last three weeks. Sigh. However (a) it's done, (b) I did a lot of other tidying up (as usual). So I'm keen to use it. Would you like to take a look? It's on branch wip/T9858-typeable-spj. Relevant log messages below. Unsurprisingly, some perf/compiler tests get a bit slower (see below). More surprisingly, there are two perf/should_run tests that get worse perf/should_run T5205 [stat not good enough] (normal) perf/should_run lazy-bs-alloc [stat not good enough] (normal) This is much more surprising: neither seems to have anything much to do with Typeable. But I can't investigate before I go on holiday. Would anyone else like to? It's all up to date wrt HEAD. Simon commit fcd18c42b6ae2e73a9f4c67dbe6d994b0f8468c8 Author: Simon Peyton Jones <simonpj(a)microsoft.com> Date: Thu Mar 26 11:06:12 2015 +0000 Remove a redndant 'return' commit a6200759fd2f3f0e901cb2092e148bf31abb08b4 Author: Simon Peyton Jones <simonpj(a)microsoft.com> Date: Mon Mar 23 14:53:53 2015 +0000 Error message wibbles These are associated with - new instance reporting - typeable changes in -ddump-simpl output - -ddump-types being a bit less verbose - some renaming of type variables in debugger output (no idea why this happens) commit fe510d911b6cf61d6884b8b42d8771d4aea3229f Author: Simon Peyton Jones <simonpj(a)microsoft.com> Date: Mon Mar 23 14:50:23 2015 +0000 Generate Typeble info at definition sites This patch implements the idea floated in #9858, namely that we should generate type-representation information at the data type declaration site, rather than when solving a Typeable constraint. However, this turned out quite a bit harder than I expected. I still think it's the right thing to do, and it's done now, but it was quite a struggle. See particularly * Note [Grand plan for Typeable] in TcTypeable (which is a new module) * Note [The overall promotion story] in DataCon (clarifies existing stuff) The most painful bit was that to generate Typeable instances (ie TyConRepName bindings) for every TyCon is tricky for types in ghc-prim etc: - we need to have enough data types around to *define* a TyCon - many of these types are wired-in Also, to minimise the code generated for each data type, I wanted to generate pure data, not CAFs with unpackCString# stuff floating about. Performance ~~~~~~~~~~~ Three perf/compiler tests start to allocate quite a bit more. This isn't surprising, because they all allocate zillions of data types, with practically no other code, esp T1969 T1969: GHC allocates 30% more T5642: GHC allocates 14% more T9872d: GHC allocates 5% more I'm treating this as acceptable. The payoff comes in Typeable-heavy code. Remaining to do ~~~~~~~~~~~~~~~ * I think that "TyCon" and "Module" are over-generic names to use for the runtime type representations used in GHC.Typeable. Better might be "TrTyCon" and "TrModule". But I have not yet done this * Add more info the the "TyCon" e.g. source location where it was defined * Use the new "Module" type to help with Trac #10068 * It would be possible to generate TyConRepName (ie Typeable instances) selectively rather than all the time. We'd need to persist the information in interface files. Lacking a motivating reason I have not done this, but it would not be difficult. Refactoring ~~~~~~~~~~~ As is so often the case, I ended up refactoring more than I intended. In particular * In TyCon, - a type *family* (whether type or data) is repesented by a FamilyTyCon - a algebraic data type (including data/newtype instances) is represented by AlgTyCon This wasn't true before; a data family was represented as an AlgTyCon. There are some corresponding changes in IfaceSyn. Also get rid of the (unhelpfully named) tyConParent. * In TyCon define 'Promoted', isomorphic to Maybe, used when things are optionally promoted; and use it elsewhere in GHC. * Each TyCon, including promoted TyCons, contains its TyConRepName, if it has one. This is, in effect, the name of its Typeable instance. * I added PatSynId, DefMethId, and ReflectionId to the IdInfo.IdDetails type. They are used for debugging only, namely to suppress excessive output in -ddump-types. * Tidy up the generation of PrelInfo.knownKeyNames * Move newImplicitBinder from IfaceEnv to BuildTyCl * PrelNames.conName renamed to dcQual for consistency with varQual, tcQual * Move mkDefaultMethodIds, mkRecSelBinds from TcTyClsDecls to TcTyDecls commit 021e6f583a1159b0a3581ab1713d560cadc2bdf7 Author: Simon Peyton Jones <simonpj(a)microsoft.com> Date: Mon Mar 23 14:32:31 2015 +0000 tcRnDeclsi can use tcRnSrcDecls I'm not sure why tcRnDeclsi didn't call tcRnSrcDecls before, but now it does. About 20 lines of code vanish. Hooray. commit 6473d110ab1aa22a5933e405b59e3f597562ce02 Author: Simon Peyton Jones <simonpj(a)microsoft.com> Date: Fri Mar 20 12:38:42 2015 +0000 Implement lookupGlobal in TcEnv, and use it This localises the (revolting) initTcForLookup function, exposing instead the more civilised interface for lookupGlobal commit 7078a11f50e3af9139dd5ceef032e89047677833 Author: Simon Peyton Jones <simonpj(a)microsoft.com> Date: Fri Mar 20 12:36:22 2015 +0000 Comments and white space commit 48512df5751a07fd503f0ba523e5504d09ee258d Author: Simon Peyton Jones <simonpj(a)microsoft.com> Date: Fri Mar 20 12:27:59 2015 +0000 Improve the error messages for class instance errors See Note [Displaying potential instances]. simonpj@cam-05-unx:~/code/HEAD-5$

2 3

How to better parallelize GHC build.
by Karel Gardas 01 Apr '15

01 Apr '15

Folks, first of all, I remember someone already mentioned issue with decreased parallelism of the GHC build recently somewhere but I cann't find it now. Sorry, for that since otherwise I would use this thread if it was on this mailing list. Anyway, while working on SPARC NCG I'm using T2000 which provides 32 threads/8 core UltraSPARC T1 CPU. The property of this machine is that it's really slow on single-threaded work. To squeeze some perf from it man really needs to push 32 threads of work on it. Now, it really hurts my nerves to see it's lazy building/running just one or two ghc processes. To verify the fact I've created simple script to collect number of ghc processes over time and putting this to graph. The result is in the attached picture. The graph is result of running: gmake -j64 anyway, the average number of running ghc processes is 4.4 and the median value is 2. IMHO such low number not only hurts build times on something like CMT SPARC machine, but also on let say a cluster of ARM machines using NFS and also on common engineering workstations which provide these days (IMHO!) around 8-16 cores (and double the threads number). My naive idea(s) for fixing this issue is (I'm assuming no Haskell file imports unused imports here, but perhaps this may be also investigated): 1) provide explicit dependencies which guides make to build in more optimal way 2) hack GHC's make depend to kind of compute explicit dependencies from (1) in an optimal way automatically 3) someone already mentioned using shake for building ghc. I don't know shake but perhaps this is the right direction? 4) hack GHC to compile needed hi file directly in its memory if hi file is not (yet!) available (issue how to get compiling options right here). Also I don't know hi file semantics yet so bear with me on this. Is there anything else which may be done to fix that issue? Is someone already working on some of those? (I mean those reasonable from the list)? Thanks! Karel

5 7

Re: traverse_
by Herbert Valerio Riedel 31 Mar '15

31 Mar '15

On 2015-03-30 at 07:05:56 +0200, Fumiaki Kinoshita wrote: [...] > I found out that (<>) (in Data.Monoid) is missing, also. It would be nice > to reexamine Prelude to export things we want to export. Fwiw, (<>) was actually left-out as it wasn't required (it's just a an alias for `mappend`), *and* to keep our options open (or at least not make it more difficult) in terms of possible migration-plans available for the case we'd be moving 'Semigroup' to base/Prelude at some point in the future.

3 4

Re: traverse_
by Herbert Valerio Riedel 31 Mar '15

31 Mar '15

On 2015-03-31 at 06:27:32 +0200, Chris Wong wrote: [...] > While it would be useful to have it in the Prelude, traverse_ doesn't > fit either of the motivations above. So I think its omission is > intended here. That is not to say that it can't be proposed for inclusion... IOW, just turn the cautious "'traverse_' is not exported from Prelude [..] Is this intentional?"-question into an explicit/proper library proposal! Cheers, hvr

1 0