- ghc-devs - Haskell.org

Heap usage of GHC increased in 7.8 vs. 7.6
by Simon Marlow 19 Apr '14

19 Apr '14

I noticed that T1969 is failing again, and decided to do a little investigation. The maximum residency when compiling T1969 with HEAD compared with 7.6.3 looks like this: 7.6.3: 10,473,920 HEAD: 13,783,536 This is with +RTS -h -i0.01 to avoid sampling errors as much as possible. The figures are pretty accurate, and the heap profiles confirm it: we're using a lot more heap now. -ddump-if-trace shows that HEAD is reading more interface files: Renamer stats: 10 interfaces read 6 type/class/variable imported, out of 1361 read 0 instance decls imported, out of 105 read 0 rule decls imported, out of 53 read vs. with 7.6.3: Renamer stats: 8 interfaces read 2 type/class/variable imported, out of 828 read 0 instance decls imported, out of 40 read 0 rule decls imported, out of 45 read The extra interface files are Control.Applicative and Control.Monad. So the question is, why are we now reading these on *every* compilation? (T1969 imports only the Prelude). Presumably this is something to do with the AMP warnings, but can't we lazily load these interfaces when they're needed? Cheers, Simon

3 4

Proposal - Foreign enum support
by Merijn Verstraaten 19 Apr '14

19 Apr '14

Cross-post to haskell-prime in case there's any interest for including this into the report's FFI specification. Proposal - Foreign enum support =============================== At the moment the FFI does not have a convenient way with interacting enums (whether proper enums or CPP defines) in C (like languages). Both enums and CPP defined enums are major parts of large C APIs and they are thus crucial to writing foreign bindings. A few examples: SDL_image defines the following enum: typedef enum { IMG_INIT_JPG = 0x00000001, IMG_INIT_PNG = 0x00000002, IMG_INIT_TIF = 0x00000004, IMG_INIT_WEBP = 0x00000008 } IMG_InitFlags; OpenCL specifies the following typedefs + CPP defined enum: typedef uint32_t cl_uint __attribute__((aligned(4))); typedef cl_uint cl_platform_info; /* cl_platform_info */ #define CL_PLATFORM_PROFILE 0x0900 #define CL_PLATFORM_VERSION 0x0901 #define CL_PLATFORM_NAME 0x0902 #define CL_PLATFORM_VENDOR 0x0903 #define CL_PLATFORM_EXTENSIONS 0x0904 OpenCL functions will return the above CPP defines as return values of type cl_platform_info. Current Solutions ----------------- In many cases someone wrapping such a C library would like to expose these enums as a simple sum type as this has several benefits: type safety, the ability to use haskell constructors for pattern matching, exhaustiveness checks. Currently the GHC FFI, as specified by Haskell2010, only marshalls a small set of foreign types and newtypes with exposed constructors of these types. As such there seem two approaches to wrap these enums: 1. Implement an ADT representing the enum and write a manual conversion function between the ADT and the corresponding C type (e.g. CInt -> Foo and Foo -> CInt). 2. Use a tool like c2hs to automatically generate the ADT and conversion function. In both cases the foreign functions are imported using the corresponding C type in their signature (reducing type safety) and the user is forced write trivial wrappers for every imported function to convert the ADT to the relevant C type and back. This is both tedious to write and costly in terms of code produced, in case of c2hs one calls "toEnum . fromIntegral" and "fromIntegral . fromEnum" for every argument/result even though this could trivially be a no-op. Worse, since c2hs uses the Enum class for it's conversion to/from C types it generates Enum instances like: instance Enum Foo where fromEnum Bar = 1 fromEnum Baz = 1337 toEnum 1 = Bar toEnum 1337 = Baz toEnum unmatched = error ("PlatformInfo.toEnum: Cannot match " ++ show unmatched) Since succ/pred and enumFromTo's default implementations assume enums convert to continuous sequence of Int this means the default generated enum instances crash. This problem could be overcome by making c2hs' code generation smarter, but this does not eliminate the tediousness of wrapping all foreign imported functions with marshalling wrappers, NOR does it eliminate the overhead of all this useless marshalling. Proposal -------- Add a new foreign construct for enums, the syntax I propose below is rather ugly and ambiguous and thereforeopen to bikeshedding, but I prefer explaining based on a concrete example. foreign enum CInt as Foo where Bar = 1 Baz Quux = 1337 Xyzzy = _ This would introduce a new type 'Foo' with semantics approximately equivalent too "newtype Foo = Foo CInt" plus the pattern synonyms "pattern Bar = Foo 1; pattern Baz = 2; pattern Quux = 1337; pattern Xyzzy = Foo _". Explicit listing of the value corresponding to a constructor should be optional, missing values should just increment by one from the previous (like C), if the initial value is missing, it should assume to start from 0. Values do not need to be contiguous. Users should be able to use these constructors as normal in pattern match (really, this mostly follows to semantics of the above pattern synonyms). The foreign import/export functionality should invisibly marshall Foo to the underlying foreign type (as is done for newtypes). I'm unsure about the support for a wildcard constructor like Xyzzy. If there is support for a wildcard, it should be optional. On the upside a wildcard means the marshalling is no longer a partial function. The downside is that it makes desugaring the use of enums in patterns harder. It seems clear that f Xyzzy = {- ... -} f Bar = {- ... -} f Baz = {- ... -} f Quux = {- ... -} Should not have the same semantics as: f (Foo _) = {- ... -} f (Foo 1) = {- ... -} f (Foo 2) = {- ... -} f (Foo 1337) = {- ... -} So in the presence of wildcards, the Foo enum can't trivially be desugared into pattern synonyms after checking exhaustiveness. Pros: 1. Foreign imports are slightly more type safe, as one can now write: foreign import ccall "someFoo.h" someFoo :: Foo -> Ptr () -> IO () Preventing users from passing an arbitrary CInt to an argument expecting a specific enum. 2. No need to write marshalling functions to/from ADT to obtain exhaustiveness checks and pattern matching 3. Cheaper as marshalling Foo to CInt is a no-op 4. toEnum/fromEnum can simply map to contiguous sequence of Int as this Int mapping is no longer used for marshalling Cons: 1. Non-standard extension of the FFI 2. Someone has to implement it 3. Wildcards constructors would present difficulties desugaring pattern matches to a simple newtype. 4. ?? What Would Need to be Done? --------------------------- 1. Parser needs to be extended to deal with parsing of enum declarations. 2. Pattern matches of an enum type need to be checked for exhaustiveness and desugared to the underlying type's representation. 3. Extend foreign imports/exports to marshall enums properly. If there's no objections I'm willing to take a stab at implementing this, although I'd probably need some help with GHC's internals (although I could bug #ghc for that). Cheers, Merijn

5 6

Cannot locally 'sync-all get'
by Daniel Fischer 19 Apr '14

19 Apr '14

Like so often, ./sync-all pull failed today in my local clones, first with "... not on a branch", after sync-all checkout master, with "fatal: Reference ... not a tree". (Aside, that nonsense didn't happen before the introduction of submodules.) The usual remedy is rm -rf ./clone git clone ./ghc ./clone cd clone ./sync-all get but today it failed with > Klone nach 'libraries/xhtml'... > Fertig. > Unterprojekt-Pfad: 'libraries/xhtml': > 'fb9e0bbb69e15873682a9f25d39652099a3ccac1' ausgecheckt fatal: Projektarchiv > '/home/dafis/GHC/./haddock.git' existiert nicht. Klonen von > '/home/dafis/GHC/./haddock.git' in Unterprojekt-Pfad 'utils/haddock' > fehlgeschlagen git failed: 256 at ./sync-all line 123. Any idea what's up? Cheers, Daniel

2 2

RE: OverloadedRecordFields
by Simon Peyton Jones 18 Apr '14

18 Apr '14

Adam I'm very happy to hear that... good stuff. I'm under water with ICFP submissions (deadline Sat). Moreover I think it is clearly too later to put this into 7.8; RC1 is out and I expect RC2 any day. So I suggest we plan to merge after 7.8 is out. Are the wiki pages up to date? Records/OverloadedRecordFields Records/OverloadedRecordFields/Implementation Records/OverloadedRecordFields/Plan The first does not point to the latter two; "Plan" may mean "Design"... I feel some rationalisation may make sense Simon | -----Original Message----- | From: Adam Gundry [mailto:adam@well-typed.com] | Sent: 24 February 2014 08:37 | To: Simon Peyton Jones | Subject: Re: OverloadedRecordFields | | Hi Simon, | | My OverloadedRecordFields branches[1,2,3] are up to date with HEAD as of | last Saturday. Validate on linux x86_64 reports only one failure, the | haddock.Cabal perf test, which might well be due to my Haddock changes, | and I will investigate. I'm not sure how to run the Haddock test suite? | | I am keen to get the code reviewed and into HEAD as soon as is | convenient, but I'm aware these are substantial changes, and don't want | to rush things. In particular, I would understand if you'd rather hold | them back until after the 7.8 final release. | | How would you like to proceed? | | Adam | | [1] https://github.com/adamgundry/ghc | [2] https://github.com/adamgundry/packages-base | [3] https://github.com/adamgundry/haddock | | | On 17/01/14 10:55, Simon Peyton Jones wrote: | > Yes that sounds ok, thanks. I'd prefer to have a write-up of what | goes wrong with the 2-parameter story, so that we don't forget. | > | > Simon | > | > | -----Original Message----- | > | From: Adam Gundry [mailto:adam@well-typed.com] | > | Sent: 17 January 2014 10:15 | > | To: Simon Peyton Jones | > | Subject: OverloadedRecordFields | > | | > | Hi Simon, | > | | > | I'm conscious that things have gone off the boil a little wrt | > | OverloadedRecordFields, partially as a consequence of the delayed | > | 7.8 release but also my lack of time for other projects since | > | starting work for Well-Typed. With that in mind, I'd like to propose | > | a plan to get back on track: | > | | > | 1. Revert to the three-parameter story, where we have | > | | > | t ~ FldTy r f => Has r f t | > | | > | rather than | > | | > | Has r f. | > | | > | The two-parameter version generates significantly worse error | > | messages, and there are some other unresolved problems, so I'm not | > | sure it is worth the minor simplification. | > | | > | 2. Roll back some of the refactoring that I've struggled to get | > | right (in particular, trying to make the generated FldTy/UpdTy | > | axioms implicitTyThings). We can always revisit this in the future | though. | > | | > | 3. Merge HEAD into my branch: I suspect this will be a bit painful | > | by now, but presumably with 7.8 imminent there won't be many major | > | changes coming for a while? | > | | > | 4. Review the proposed changes with you and fix any show-stopping | > | problems. | > | | > | 5. Merge into HEAD after 7.8 is released. | > | | > | Does this sound plausible? I'm happy to Skype if you like. | > | | > | Cheers, | > | | > | Adam | > | | > | P.S. I'm not sure if Andrew Kennedy has mentioned it to you, but | > | Neil Ghani has got me some funding to work with them both on units | > | of measure for Haskell. We are still sorting out the details, but I | > | hope it might be possible to work on some kind of plugin mechanism | > | for GHC's constraint solver, along the lines that Iavor has been | > | investigating, if that would be of interest? | > | | > | -- | > | Adam Gundry, Haskell Consultant | > | Well-Typed LLP, http://www.well-typed.com/ | > | | | -- | Adam Gundry, Haskell Consultant | Well-Typed LLP, http://www.well-typed.com/

4 4

strip related failure [was: Re: solaris-x86-head (Solaris/x86 HEAD (Karel Gardas)), build 32, Failure]
by Karel Gardas 18 Apr '14

18 Apr '14

Folks, last two/three builds on Solaris builder fails due to a reason that someone added (probably) stripping of installed libraries. My bet is on 8992d5269804b727fb77249511e89df678526907 -- hence ccing you Herbert, but I'm not sure... The problem is that invoked strip command line is incompatible with Solaris' strip which then fails with: strip: --strip-unneeded: cannot open file: No such file or directory the problem is usage of --strip-unneeded command-line argument? It looks like this argument is supported by just GNU strip. Do we really need to use it? If not, removing this may solve the issue. If yes, then I'll need to come with some workaround for Solaris... Detailed log of failure is at the bottom of http://haskell.inf.elte.hu/builders/solaris-x86-head/32/20.html Thanks! Karel On 04/18/14 05:03 AM, Builder wrote: > solaris-x86-head (Solaris/x86 HEAD (Karel Gardas)), build 32 > > Build failed > Details: http://haskell.inf.elte.hu/builders/solaris-x86-head/32.html > > git clone | Success > create mk/build.mk | Success > get subrepos | Success > repo versions | Success > touching clean-check files | Success > setting version date | Success > booting | Success > configuring | Success > creating check-remove-before | Success > compiling | Success > creating check-remove-after | Success > compiling testremove | Success > simulating clean | Success > checking clean | Success > making bindist | Success > making srcdist | Success > uploading bindist | Success > uploading srcdist | Success > uploading tarball source | Success > testing bindist | Failure: Just (ExitFailure 2) > > Build failed > Details: http://haskell.inf.elte.hu/builders/solaris-x86-head/32.html > > > > > _______________________________________________ > ghc-builds mailing list > ghc-builds(a)haskell.org > http://www.haskell.org/mailman/listinfo/ghc-builds

3 4

GHC's performance
by Simon Peyton Jones 18 Apr '14

18 Apr '14

Friends The thread below concerns GHC's performance. I'm writing to ask for your help. In developing GHC we always run 'validate', which runs a lot of regression tests. A few of those are performance tests, but because we do so frequently, none of these performance tests run for long, and none depend on other packages. And they tend to test something very specific. What we lack is a sustained effort to track a) The performance of GHC itself b) The performance of GHC-compiled programs At GHC HQ we *aspire* to track this stuff, but in practice we simply don't. Bug-fixing, portability, etc etc always ends up taking priority. But it's a pity that we don't. For example, Michal's comment below (that his GHC-compiled program has run 10-20% faster with each release of GHC from 6.12) is fantastic -- but we have no data to back it up, or know whether it's just Michael, or more widely true. I also suspect that sometimes we regress and don't know it. People do report this (e.g. #8971), but it's a bit random. Again, it would be great to have a more systematic way to know. In many cases it might be easy to fix; but we can only fix if we know. We have the nofib suite, and we take that very seriously, but it is showing its age, uses no advanced features, and I'm not sure how representative it is any more. Johan Tibell and Bryan O'Sullivan agreed to become GHC's Performance Tsars a year or two ago, with a view to focusing on (b) at least, but they are both extremely busy. And I don't think they even attempt to focus on (a). So I'm wondering: would anyone like to help here? It would mean * Soliciting and gathering together some more substantial benchmarks * Gathering performance numbers * Yelling quickly if the numbers go bad. * Investigating why (e.g. no one has seriously profiled GHC itself for a while, both space and time. I bet there are improvements to be had there.) Maybe it could be part of the new buildbot team's work? Certainly it'd make sense to use the same nightly-build infrastructure. Anyway, I'm advertising that there's an un-met need, and I'd love some help. Thanks Simon | -----Original Message----- | From: Michal J. Gajda [mailto:mjgajda@gmail.com] | Sent: 10 April 2014 07:26 | To: ghc-devs(a)haskell.org; Sergei Trofimovich; Manuel Chakravarty; Simon | Peyton Jones | Subject: Re: ghc-7.8-rc2 in -O2 mode eats all stack and RAM on pandoc- | ,citeproc / highlighting-kate | | Dear Devs, | | On 04/10/2014 09:09 AM, ghc-devs-request(a)haskell.org wrote: | > Filed the reproducer as a new ticket: | > https://ghc.haskell.org/trac/ghc/ticket/8980 | > | > [ Looks like highlighting-kate asks to be added to | > compiler performance benchmarks (are there such ones?) | > It tends to stress ghc all the time: | > http://hackage.haskell.org/trac/ghc/ticket/3664 | > ] | Please consider adding hPDB too, if you want to stress the optimizer. | It shows GHC optimizer at its best, with at least 10-20% improvement in | every major version of | the compiler since 6.12. Unfortunately at cost of very long compile | times. | Please let me know if I should submit a driver code for automatic | benchmarks it (it is in hPDB-examples.) Thanks! | >> > SpecConstr is too aggressive: it sometimes blows up the program | badly and we have no good | solution. See Trac #7898, #7068, #7944, #5550, #8836. | And #8960, where GHC runs out of memory. (Only in 7.8.) Should be easy | to reproduce by just `cabal install hPDB`. | >> > I notice that the latter three are actually fixed in 7.8, so worth | trying that. If it | still fails, do add instructions to reproduce to one of the above open | tickets, or make a new one. | >> > | >> > Meanwhile you can use -fno-spec-constr to simply switch it off for | offending modules. That should get you going. | -- | Best regards | Michal

4 6

Git problem
by Simon Peyton Jones 17 Apr '14

17 Apr '14

Friends I'm stuck with Git. Help please! Simon I tried to do a 'git stash pop', having just done a 'git pull' from the master repo. I got this: simonpj@cam-05-unx:~/code/HEAD-2$ git stash pop warning: Failed to merge submodule utils/haddock (commits don't follow merge-base) Auto-merging utils/haddock CONFLICT (submodule): Merge conflict in utils/haddock Auto-merging compiler/typecheck/TcRnMonad.lhs Now git status says simonpj@cam-05-unx:~/code/HEAD-2$ git status # On branch master # Your branch is ahead of 'origin/master' by 2 commits. # # Changes to be committed: # (use "git reset HEAD <file>..." to unstage) # # modified: compiler/typecheck/TcMType.lhs # modified: compiler/typecheck/TcRnMonad.lhs # modified: compiler/typecheck/TcSimplify.lhs # modified: compiler/typecheck/TcUnify.lhs # # Unmerged paths: # (use "git reset HEAD <file>..." to unstage) # (use "git add/rm <file>..." as appropriate to mark resolution) # # both modified: utils/haddock The modified files are right. It's the "both modified utils/haddock" that is messing me up. I'm not modifying haddock! I just want to say "take the master haddock", but I don't know how. I tried 'git checkout utils/haddock' but that just led do `git status' saying # modified: utils/haddock (new commits)

4 4

Link failure
by Simon Peyton Jones 17 Apr '14

17 Apr '14

On 64-bit Linux I'm getting this in the link step for the template-haskell library: /usr/bin/ld: libraries/template-haskell/dist-install/build/Language/Haskell/TH/Syntax.dyn_o: relocation R_X86_64_PC32 against undefined symbol `templatezmhaskell_LanguageziHaskellziTHziSyntax_OccName_closure' can not be used when making a shared object; recompile with -fPIC /usr/bin/ld: final link failed: Bad value collect2: ld returned 1 exit status make[2]: *** [libraries/template-haskell/dist-install/build/libHStemplate-haskell-2.10.0.0-ghc7.9.20140416.so] Error 1 make[1]: *** [all_libraries/template-haskell] Error 2 make[1]: Leaving directory `/5playpen/simonpj/HEAD-1' My compiler is not vanilla HEAD, but I have changed only stuff in the Core simplifier, nothing in the back end. Does anyone have the faintest idea what is going on? It's extremely annoying because I can't make progress without working around it somehow. Thanks Simon

1 1

ghc-builder: builder-client: fd:13: hGetLine: resource exhausted (Resource temporarily unavailable)
by Alain O'Dea 17 Apr '14

17 Apr '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I get the following error every time I run the ghc-builder: builder-client: fd:13: hGetLine: resource exhausted (Resource temporarily unavailable) This appears to be the kernel returning EAGAIN (non-fatal) on an IO request. I'm not sure why it crashes the builder-client. I would expect hGetLine to retry in this case. I'm running this on SmartOS, which is an Illumos (OpenSolaris 10 compatible) distro. Here's a log of an entire run: https://gist.github.com/AlainODea/12a74a05b52c334614d9 Here's a snippet around the problem: Running "touching clean-check files" Running "setting version date" Running "booting" builder-client: fd:13: hGetLine: resource exhausted (Resource temporarily unavailable) [2014-04-17 02:21:08] [smartos-x86-head] Sending: "UPLOAD 14 1" Received: "202 Send name" Received: "202 Send subdir" Received: "202 Send program" Has anyone seen something like this before? Thanks, Alain -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJTT0ApAAoJEP0rIXJNjNSAFbcH/29X7XvvMjGg/DWocap7D40f QOrVNTAipRcRSvEeCQh61hAAouabC2MkXts/8qM171xSghHArlYqe5XLkeKUDpo+ RiuyQER2t5fiZU1u8TywdMOm0NegxySiltEKooKpWzIzDzZHX7ggDUQlWTVzFgmR S50OsF8z9i3t8EF4N23J/Y78qca97iTn+Rw9qYl8Nc4vbhd+1Mqrsd25KDBK73Vk 5Hzy+HFy6yaJIEF3Rpda1Wr1CYKT4mwW6wAoTOeAK3obogwWX1mBzpV4MDt/5NBZ 9qet+2n59jkk0aaXJszXqNz4T0MTpMXgummjiLghyxtFJu71bvv3Xs6FXwgP23I= =6fKZ -----END PGP SIGNATURE-----

1 0

7.8.2 bugs and beyond
by Austin Seipp 16 Apr '14

16 Apr '14

Hello all, 7.8.1 is out. Yay! Now, on to the next thing. Here's the very preliminary 7.8.2 buglist: - https://ghc.haskell.org/trac/ghc/query?group=status&milestone=7.8.2 Note that this list has effectively received zero curation so far. I plan on fixing that soon, and in the process I am probably going to touch a lot of trac tickets (so I apologize in advance for the mail spam, because y'all are probably going to get a lot of it). This will include re-prioritizing these bugs, so take things with a grain of salt right now. Note that the bug tracker now has 7.8.1 as the default version - if you're filing bugs, please be sure to try it first of course. :) And don't forget to set the milestone to 7.8.2 if you think it qualifies for a fix. There are already a couple of things in the merge queue. There's currently no expected date for 7.8.2. Should it arrive quickly? Or slowly? Simon, Simon and I are inclined to just let things play out for a while of course, which is the traditional way things are done. But this release window has been especially awkward, so we'll see what happens. This of course leaves timeline questions for 7.10 open - but before we decide that, now's a time to just get some hacking done I suppose... -- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

5 6