July 2014 - ghc-devs - Haskell.org

Haskell Platform 2014.2.0.0 Release Candidate 2
by Mark Lentczner 29 Jul '14

29 Jul '14

The long anticipated Haskell Platform 2014.2 release, including GHC 7.8.3, and numerous updated packages, is almost here! We have created "Release Candidate 2" installers for OS X and Windows, and believe, barring show stopper issues, creepers exploding, or other bizarre phenomenon, these will likely be blessed as the final in a few days time. If you would like to be an early adopter, please try 'em out... - source tarball: haskell-platform-2014.2.0.0-RC2.tar.gz <http://www.ozonehouse.com/mark/platform/haskell-platform-2014.2.0.0-RC2.tar…> - source repo: haskell/haskell-platform at 2014.2.0.0-RC2 <https://github.com/haskell/haskell-platform/tree/2014.2.0.0-RC2> - windows 32bit: hskellPlatform-2014.2.0.0-i386-RC2-setup.exe <http://www.ozonehouse.com/mark/platform/hskellPlatform-2014.2.0.0-i386-RC2-…> - windows 64bit:hskellPlatform-2014.2.0.0-x86_64-RC2-setup.exe <http://www.ozonehouse.com/mark/platform/hskellPlatform-2014.2.0.0-x86_64-RC…> - os x 64bit: Haskell Platform 2014.2.0.0 64bit RC2.signed.pkg <http://www.ozonehouse.com/mark/platform/Haskell%20Platform%202014.2.0.0%206…> - travis-ci build: haskell/haskell-platform <https://travis-ci.org/haskell/haskell-platform> *Notes for RC2, since RC1:* *Windows* *Extra thanks to Randy Polen for burning the midnight-oil to get ths out* - removed unneeded python (et al) files from the GHC bindist for 64-bit Windows (referenced in GHC ticket #9014 <https://ghc.haskell.org/trac/ghc/ticket/9014>) - added HTML "view source" pages for the GHC packages that was missing from the GHC bindist for both 32- and 64-bit Windows. *Mac OS X* *If you installed RC1, you can remove first with the command *sudo uninstall-hs only 7.8.3 --remove *Run it without *--remove* to see what it will do before running, it if you like. In theory you can just install this one right on top of RC1.... but hasn't been tested.* - file layout on the Mac... improved slightly (again). In particular, executables are now installed directly in $prefix/bin dirs, rather than within the package dir - fix the bug with haddock master index not being updated correctly - works on 10.6! (and 10.7, 10.8, and 10.9) with gcc or clang based Xcodes - works on 10.10!!! (Yosemite, developer preview 4 release) *Source tarball* - missing sources in hptool now present - platform.sh improved somewhat (in particular, handles host cabal being pre-sandbox) *Timetable* - These can "soak" amongst the intreped on these lists for a few hours. - On Monday evening (PST) I'll announce to haskell-cafe and reddit - End of next week (from my vacation, I'll point out), we'll declare success and ship. — Mark *SHA-256 sums:* 62f39246ad95dd2aed6ece5138f6297f945d2b450f215d074820294310e0c48a Haskell Platform 2014.2.0.0 64bit RC2.signed.pkg 7c7d3585e89e1407461efea29dcaa9628c3be3c47d93a13b5a4978046375e4fd haskell-platform-2014.2.0.0-RC2.tar.gz 6eedd76aafb266d9a09baff80cd2973498ab59195c771f7cd64425d40be29c49 hskellPlatform-2014.2.0.0-i386-RC2-setup.exe b22115ed84d1f7e747d7f0b47e32e1489e4a24613d69c91df4ae32052f88b130 hskellPlatform-2014.2.0.0-x86_64-RC2-setup.exe

2 3

.gitignore of tests
by Dr. ERDI Gergo 29 Jul '14

29 Jul '14

Hi, Is there a good reason we have a single monolithic .gitignore file in testuiste/ instead of one per directory? It just feels so unnecessarily burdensome to maintain it compared to just putting four lines in a new .gitignore file... Bye, Gergo

3 2

redundant rts/StgPrimFloat decodeDouble?
by Herbert Valerio Riedel 29 Jul '14

29 Jul '14

Hello *, While working on integer-gmp2 I noticed that there seems to be redundant code between rts/StgPrimFloat.c and integer-gmp. Specifically, there's a void __decodeDouble_2Int (I_ *man_sign, W_ *man_high, W_ *man_low, I_ *exp, StgDouble dbl); C implementation which does some low-level bit-fiddling on the IEEE representation instead of just using the more portable standard frexp() function, like I'm doing in integer-gmp2, e.g.: HsInt integer_gmp_decode_double (const HsDouble x, HsInt64 *const mantisse) { if (x) { int exp = 0; *mantisse = (HsInt64)scalbn(frexp(x, &exp), DBL_MANT_DIG); return exp-DBL_MANT_DIG; } else { *mantisse = 0; return 0; } } A similiar operation exists in integer-gmp/cbits/float.c So here's my questions: 1) Is there any reason/value in doing low-level IEEE bit-fiddling instead of just delegating to the C99/POSIX compliant frexp(3) operations? 2) Specifically, `decodeDouble_2Int` seems to be dead-code. I'd like to instead move the HsInt integer_gmp_decode_double (const HsDouble x, HsInt64 *const mantisse); which I currently import as #if WORD_SIZE_IN_BITS == 32 foreign import prim "integer_gmp_cmm_decode_double" cmm_decode_double :: Double# -> (# Int64#, Int# #) #elif WORD_SIZE_IN_BITS == 64 foreign import prim "integer_gmp_cmm_decode_double" cmm_decode_double :: Double# -> (# Int#, Int# #) #endif based on the size of `Int#` into rts/PrimOps.cmm & rts/StgPrimFloat.c as that may allow me to avoid any C-- in integer-gmp2 altogether (as I only need to go via C-- to return an unboxed pair) Cheers, hvr

1 0

Re: Dashboard
by Alexander Pakhomov 29 Jul '14

29 Jul '14

Hi Joachim, I think automatic regression notification at least to the author is a good idea. Probably I can do it in a nearest time. Unfortunately, right now I fail to get your code up. Also I believe it is a good style to check commits for regressions before pushing them. But maybe GHC community is less performance oriented. 25.07.2014, 01:41, "Joachim Breitner" <mail(a)joachim-breitner.de>: > Hi Alexander, > > Am Donnerstag, den 24.07.2014, 19:22 +0400 schrieb Alexander Pakhomov: >> I've been informed you created http://ghcspeed-nomeata.rhcloud.com. I >> believe it is necessary for any serious perfomance work. >> Do you need some help? What have you done and what problems are >> unsolved? > thanks for your interest. > > One way to help is to monitor the page for regressions, and notify > whoever caused it. I’m doing that from time to time, relying on the > Latest Results summary. > > Then I don’t find the codespeed software to be perfect. The thread at > https://groups.google.com/forum/#!topic/codespeed/yY5-kPrcG94 discusses > some of the things I don’t like. So if you feel like hacking Python, > just hack away on it and I’ll happily accept pull requests at > https://github.com/nomeata/codespeed (branch ghc), or directly upstream. > > Eventually, the whole setup should be moved to some dedicated and > official hardware, but I’m not in a hurry with this. It’s been running > for just a week or two, and I’d like to observe how its behaving. > > Greetings, > Joachim > > PS: I prefer to discuss these things on the appropriate mailing list and > in public, if only for the archive. If you agree, simply reply to the > ghc-dev list (quoting in full) and I’ll read your reply there. > > -- > Joachim “nomeata” Breitner > mail(a)joachim-breitner.de • http://www.joachim-breitner.de/ > Jabber: nomeata(a)joachim-breitner.de • GPG-Key: 0xF0FBF51F > Debian Developer: nomeata(a)debian.org

2 1

Status updates
by Austin Seipp 29 Jul '14

29 Jul '14

Hello *, Here are some notes on what I've done in the past week, and what I plan on doing going forward: - First and foremost, I made a bunch of improvements to Phabricator/Trac. Now, most noticeably: 1) Phabricator has a field for GHC trac tickets. You can specify this when you run 'arc diff', and it will appear on the Differential Revision. By default if you run 'arc diff', this field appears at the bottom, below 'Subscribers:'. 2) Phabricator now supports Trac markup syntax - if you say '#7602' in a trac ticket, this turns into a hyperlink with the text "Trac #7602" and links to the ticket. 3) The IRC bot on #ghc can now look up Trac tickets for you (this is mostly only relevant to those of us there) - The final piece of this whole story is actually getting Phabricator to *comment* on Trac. This is almost done, but due to the fact I'm writing PHP (and thus have absolutely no idea what I'm doing) the code still seems slightly broken somewhere since it's one of the larger parts of the Phabricator integration. Hopefully I can finish this soon. I will say though - thank you Herbert for helping me figure out the Trac XML/RPC plugin, and getting it working for me! - Another Phabricator related thing: I've begun working on a better Harbormaster build backend - it will actually report failures, support concurrent builds, and give you stderr in log results! Yay! - I took some time to further clean up the Git wiki page following last week, but only made minor progress here. - The patch queue is, in fact, being drained as we speak! I have a whole bunch of incoming patches from contributors running under ./validate as I write this, including some cleanup commits too (to detab/whitespace things). Not all of them, but a good chunk I could fit in that were low-impact and correct. - I have not finished AMP. :( This is quickly going to be my #1 priority before fixing any bugs because at this point, it's holding up other `base` improvements (Edward K, I'm sure, is getting a bit fiddly about this getting done :) This week: - Before fixing *any* new bugs myself, I'm going to finish AMP, because it's a blocker for others including Edward, Simon, and future base improvements. I'm thinking I may just do this before any more Phabricator improvements. - I will hopefully finish the Trac and Phabricator setup soon. at the very least I want to get the comment updates working. I'll probably continue with the build system shenannigans sometime after. - The wiki pages and Git stuff still need more cleanup to streamline them. This will be happening when I get a chance (Herbert already improved some things I touched last week - Thanks!) -- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

2 1

Trac
by Simon Peyton Jones 28 Jul '14

28 Jul '14

Trac is taking minutes to display a page. Also spam is still coming Simon

2 1

RE: [commit: ghc] master: Module reexports, fixing #8407. (7f5c1086)
by Simon Peyton Jones 28 Jul '14

28 Jul '14

Edward Great stuff. Is this documented somewhere, notably in http://www.haskell.org/ghc/docs/latest/html/users_guide/packages.html for GHC, and somewhere in Cabal? And perhaps somewhere on the wiki https://ghc.haskell.org/trac/ghc/wiki/Commentary/Packages Thanks Simon | -----Original Message----- | From: ghc-commits [mailto:ghc-commits-bounces@haskell.org] On Behalf Of | git(a)git.haskell.org | Sent: 26 July 2014 02:08 | To: ghc-commits(a)haskell.org | Subject: [commit: ghc] master: Module reexports, fixing #8407. | (7f5c1086) | | Repository : ssh://git@git.haskell.org/ghc | | On branch : master | Link : | http://ghc.haskell.org/trac/ghc/changeset/7f5c10864e7c26b90c7ff4ed09d00 | c8a09aa4349/ghc | | >--------------------------------------------------------------- | | commit 7f5c10864e7c26b90c7ff4ed09d00c8a09aa4349 | Author: Edward Z. Yang <ezyang(a)cs.stanford.edu> | Date: Fri Jul 4 17:01:08 2014 +0100 | | Module reexports, fixing #8407. | | The general approach is to add a new field to the package database, | reexported-modules, which considered by the module finder as | possible | module declarations. Unlike declaring stub module files, multiple | reexports of the same physical package at the same name do not | result in an ambiguous import. | | Has submodule updates for Cabal and haddock. | | NB: When a reexport renames a module, that renaming is *not* | accessible | from inside the package. This is not so much a deliberate design | choice | as for implementation expediency (reexport resolution happens only | when | a package is in the package database.) | | TODO: Error handling when there are duplicate reexports/etc is not | very | well tested. | | Signed-off-by: Edward Z. Yang <ezyang(a)cs.stanford.edu> | | Conflicts: | compiler/main/HscTypes.lhs | testsuite/.gitignore | utils/haddock | | | >--------------------------------------------------------------- | | 7f5c10864e7c26b90c7ff4ed09d00c8a09aa4349 | compiler/main/DynFlags.hs | 1 + | compiler/main/Finder.lhs | 25 +++-- | compiler/main/GHC.hs | 12 ++- | compiler/main/HscTypes.lhs | 6 +- | compiler/main/PackageConfig.hs | 4 + | compiler/main/Packages.lhs | 109 | ++++++++++++++++----- | ghc/InteractiveUI.hs | 8 +- | libraries/Cabal | 2 +- | .../Distribution/InstalledPackageInfo/Binary.hs | 8 ++ | testsuite/.gitignore | 8 ++ | testsuite/tests/cabal/Makefile | 15 +++ | testsuite/tests/cabal/all.T | 6 ++ | testsuite/tests/cabal/cabal05/Makefile | 69 +++++++++++++ | .../{driver/T3007/A => cabal/cabal05}/Setup.hs | 0 | testsuite/tests/cabal/{cabal03 => cabal05}/all.T | 4 +- | .../tests/cabal/cabal05/p/LICENSE | 0 | testsuite/tests/cabal/cabal05/p/P.hs | 3 + | testsuite/tests/cabal/cabal05/p/P2.hs | 1 + | .../{driver/T3007/A => cabal/cabal05/p}/Setup.hs | 0 | testsuite/tests/cabal/cabal05/p/p.cabal | 11 +++ | .../tests/cabal/cabal05/q/LICENSE | 0 | testsuite/tests/cabal/cabal05/q/Q.hs | 4 + | .../{driver/T3007/A => cabal/cabal05/q}/Setup.hs | 0 | testsuite/tests/cabal/cabal05/q/q.cabal | 29 ++++++ | .../tests/cabal/cabal05/r/LICENSE | 0 | testsuite/tests/cabal/cabal05/r/R.hs | 11 +++ | .../{driver/T3007/A => cabal/cabal05/r}/Setup.hs | 0 | testsuite/tests/cabal/cabal05/r/r.cabal | 32 ++++++ | .../tests/cabal/cabal05/s/LICENSE | 0 | testsuite/tests/cabal/cabal05/s/S.hs | 18 ++++ | .../{driver/T3007/A => cabal/cabal05/s}/Setup.hs | 0 | testsuite/tests/cabal/cabal05/s/s.cabal | 11 +++ | testsuite/tests/cabal/ghcpkg07.stdout | 11 +++ | .../{test4.pkg => recache_reexport_db/a.conf} | 20 ++-- | testsuite/tests/cabal/{test4.pkg => test7a.pkg} | 20 ++-- | testsuite/tests/cabal/test7b.pkg | 17 ++++ | utils/ghc-cabal/ghc-cabal.cabal | 3 +- | utils/ghc-pkg/Main.hs | 55 ++++++++++- | utils/ghc-pkg/ghc-pkg.cabal | 4 +- | utils/ghctags/ghctags.cabal | 3 +- | utils/haddock | 2 +- | 41 files changed, 453 insertions(+), 79 deletions(-) | | Diff suppressed because of size. To see it, use: | | git diff-tree --root --patch-with-stat --no-color --find-copies- | harder --ignore-space-at-eol --cc | 7f5c10864e7c26b90c7ff4ed09d00c8a09aa4349 | _______________________________________________ | ghc-commits mailing list | ghc-commits(a)haskell.org | http://www.haskell.org/mailman/listinfo/ghc-commits

2 2

Understanding core2core optimisation pipeline
by Jan Stolarek 28 Jul '14

28 Jul '14

Devs, I'm trying to understand how the core2core pipeline works. Sadly, we don't have a wiki page about this so the "only" source of information are the papers and the source code. Papers give pretty much detail about each transformation in separate but none of the papers gives a comprehensive and up-to-date overview of how the whole pipeline is structured. My questions are based on reading the user documentation and the following papers: [1] - "The Glasgow Haskell Compiler" from The Architecture of Open Source Application, vol. 2 [2] - "Compilation by Transformation in Non-Strict Functional Languages", PhD by Santos [3] - "Secrets of the Glasgow Haskell Compiler inliner" [4] - "A transformation-based optimiser for Haskell" [5] - "Modular, Higher-Order Cardinality Analysis in Theory and Practice" [6] - "Let-floatig: moving bindings to give faster programs" [7] - "Playing by the Rules: Rewriting as a practical optimisation technique in GHC" I know there are several papers missing from this list, eg. "Constructed Product Result Analysis for Haskell" or "Call-pattern specialisation for Haskell programs". The reason is that these optimisations are beyond the scope of what I'm doing at the moment (or so I believe). This mail basically asks just one question: what is the order of optimizations pefromed on Core? Since this question is pretty big and general I've separated it into smaller questions that arose from reading the above papers, documentation, and experimenting with GHC. Now the detailed questions: 1. What is the difference between a "simplifier iteration" and "simplifier phase"? Section 7.20.6.5 of the user guide mentions phases but I believe that iterations are not explained anywhere. My best guess, expressed in pseudo-code, is this (sorry about the imperative style): foreach (i in iterations) { // some optimisations here? foreach (p in phases) { //...optimisations here } // some optimisations here? } 1a. What is the default maximum iterations count? User documentation does not specify that. 2. How can I observe the effects of `-ddump-simpl-phases`. I tried compiling several different programs and this flag seems to have no effect (ie. nothing gets printed). 3. Cardinality anlaysis and inlining: cardinality analysis can determine that a let binding is used exactly once. Can the inliner re-use this information from the cardinality analysis or does it recompute it per [3], section 3.1? 4. I've compiled a sample program using `-dverbose-core2core` and got the following phases: - Desugar (after optimization) - Simplifier (Phase = InitialPhase [Gentle]) - Specialise - Levels added - Float out - Float inwards - Simplifier (Phase = 2 [main]) - Simplifier (Phase = 1 [main]) - Simplifier (Phase = 0 [main]) - Demand analysis - Worker Wrapper binds - Simplifier (Phase = 0 [post-worker-wrapper]) - Levels added - Float out - Common sub-expression - Float inwards - Simplifier (Phase = 0 [final]) - Tidy Core - CorePrep This raises lots of questions: 4a. The first phase is "Desugar (after optimization)". What optimizations are performed during desugaring? 4b. I'm not sure whether I'm looking at a single iteration of core2core transformation or at multiple ones. Some passes are performed several times (Float out, Float inwards), which suggests that there might be many iterations here. On the other hand simplifier phases are decreasing towards 0, which looks as if it was one core2core iteration. My assumption here is that every time a new core2core iteration starts the simplifier phases are counted anew from 2 towards 0. Is that correct? 4c. Why are there several 0 phases of the Simplifier? I find it confusing. 4d. I understand that some passes can be enabled or disabled using command-line options. Can the decission to run some passes be made dynamically by the compiler (eg. to run extra simplifier passes)? 4e. Are there more phases that could appear here, ie. they were ommited with -O? 4f. "Levels added" pass before the "Float out" pass: my guess is that this is preparation for the full laziness transform. So, is full laziness performed as part of "Float out" pass? A general note is that I am confused by many Simplifier passes being interleaved with other passes. I expected that simplifier phases will grouped into a single pass, as speculated in question 1. 5. What optimizations *exactly* are performed by the Simplifier? I assume that most of what's described in chapter 3 of [2]: beta reduction, let elimination, case elimination, case floating, constant folding and eta expansion. I'm not sure about floating let outwards and inwards - [1], pg. 7, says these are in a pass separate from the simplifier. `-dverbose-core2core` seems to confirm that since it reveals separate "Float out" and "Float inwards" passes. 6. [4], pg. 31, mentions the Deforestation optimisation. Is everything described in that "Deforestation" section subsumed by cardinality analysis ([5], end of section 2.1 and section 7.1)? If not then when is deforestation performed? 7. [5], section 6.1 says: "We run the analysis twice, once in the middle of the optimisation pipeline and once near the end". When exactly in the middle of the pipeline? Between which passes? This does not show up with `-dverbose-core2core` (or at least it is not explicitly named). 8. How does the rules rewriting fit into the picture? Section 7.20.6.5 of the User Guide and section 4.1 of [7] explain the interaction between rules and inlining and my guess is that both are performed by the Simplifier. Again the "simplifier phases/iterations" distinction puzzles me as to what exactly is happening when. Within a single phase is the inlining happening before rewriting or vice versa? I know that all of the above questions can be answered by looking at the source code for sufficiently long. This is actually what I'm planning to do next but if anyone could help me by answering some of these questions this would certainly save me some time. My plan is to gather up the answers on a wiki page. Janek

2 2

phabricator issue with git submodules.
by Karel Gardas 27 Jul '14

27 Jul '14

Hi, just fixing few warning issues on Solaris/x86. The changes spread over main ghc tree and libraries/primitive and libraries/unix. I already commited changes and pushed to my github.com's forks of libraries/primitive and libraries/unix. The git status looks then: $ git status On branch master Your branch is ahead of 'origin/master' by 2 commits. (use "git push" to publish your local commits) Changes not staged for commit: (use "git add <file>..." to update what will be committed) (use "git checkout -- <file>..." to discard changes in working directory) modified: libraries/primitive (new commits) modified: libraries/unix (new commits) no changes added to commit (use "git add" and/or "git commit -a") and yet phabricator still complains about it: $ arc diff You have unstaged changes in this working copy. Working copy: /export/home/karel/vcs/ghc-src/validate-fixes/ Unstaged changes in working copy: libraries/primitive libraries/unix Do you want to amend these files to the commit? [y/N] Usage Exception: Stage and commit (or revert) them before proceeding. I pressed enter in question above. Is that a known issue or am I doing something wrong here? Thanks! Karel

2 3

Extending ghc-pkg to handle installed package IDs
by Edward Z. Yang 26 Jul '14

26 Jul '14

Hello all, Since we're relaxing the constraint that a package ID correspond to exactly one installed package in the package database, it will now sometimes be necessary to query the package database using ghc-pkg for a specific installed package ID. At the moment, none of the commands support this, and I'd like to request comments on proposed ways of adding this functionality. Here are the proposals: 1. All command line modes stay the same, but anywhere a package ID was previously accepted, now we also accept an installed package ID. We distinguish the two by using the IPID parser from Cabal, and falling back the the package ID parser (assuming that version tags are not used, this is unambiguous). 2. Add a new flag --id which changes the meaning of a package ID field to be an installed package ID field. 3. Add new variants of all commands such as 'ghc-pkg unregister-id' which accept package IDs. 4. ... 5. Profit!! Cheers, Edward

1 0