July 2014 - ghc-devs - Haskell.org

gcc vs. clang builds of 7.8.3 on OS X
by Mark Lentczner 13 Jul '14

13 Jul '14

In building the OS X bindist for 7.8.3, I had to choose which of several ways to build it. In particular, I could build it with a newere Xcode, which uses clang, or an older Xcode which uses gcc. I decided to nofib benchmark the variations and see before I released. Here is what I found... I compared two candidate builds: - x86_64 architecture - targeted at OS X 10.7 and later - one built with Xcode 5.1 on 10.9, which uses clang - one built with Xcode 4.5 on 10.8, which uses gcc I installed both bindists, side-by-side on the same machine: a 10.9 machine, with Xcode 5.1, which uses clang. The machine is a MacMini, 2.5GHz Intel Core i5 (dual core, reports as 4 cpus). Summary: - clang build was always faster - non-threaded was -3.2% run-time - threaded was -7.3% run-time - clang's improvement in GC run-time was better than -10% - clang builds were significantly bigger You can find the details here: - analysis-Silver-10.9-gcc-vs-clang.html <http://www.ozonehouse.com/mark/platform/analysis-Silver-10.9-gcc-vs-clang.h…> - analysis-Silver-10.9-gcc-vs-clang-threaded.html <http://www.ozonehouse.com/mark/platform/analysis-Silver-10.9-gcc-vs-clang-t…> The only concern is that the binary sizes were significantly bigger: +230% - I haven't investigated more, but I'm wondering if nofib doesn't strip the binaries before measuring, and perhaps clang's debugging info is much greater? Next up... we are evaluating a bindist built with the HPC Mac OS X gcc compiler (based on gcc 4.9)... and preliminary results are looking even better! Stay tuned... - Mark

4 9

Proposal: require Haddock comment for every new top-level function and type in GHC source code
by Johan Tibell 13 Jul '14

13 Jul '14

Hi! I found myself exploring new parts of the GHC code base the last few weeks (exciting!), which again reminded me of my biggest frustration when working on GHC: the lack of per-function/type (Haddock) comments. GHC code is sometimes commented with "notes", which are great but tend to (1) mostly cover the exceptional cases and (2) talk about the implementation of a function, not how a caller might use it or why. Lack of documentation, in GHC and other software projects, usually has (at least) two causes: - Programmers comment code they think is "complex enough to warrant a comment". The problem is that the author is usually a poor judge of what's complex enough, because he/she is too familiar with the code and tends to under-document code when following this principle. - Documenting is boring and tends to have little benefit the person writing to documentation. Given lack of incentives we tend to document less than we ought to. I've only seen one successful way to combat the lack of documentation that stems from the above: have the project's style guide mandate that top-level functions and types (or at least those that are exported) have documentation. This works well at Google. Anecdote: we have one code base inside Google that was until recently exempt from this rule and documentation is almost completely absent in that code base, even though hundreds of engineers work on and need to understand it every day. This breeds institutional knowledge problems i.e. if the author of a core piece of code leaves, lots of knowledge is lost. *Proposal: *I propose that we require that new top-level functions and types have Haddock comments, even if they start out as a single, humble sentence. I've found that putting even that one sentence (1) helps new users and (2) establishes a place for improvements to be made. There's a strong "broken window" effect to lack of comments, in that lack of comments breeds more lack of comments as developers follow established practices. We should add this requirement to the style guide. Having it as a written down policy tends to prevent having to re-hash the whole argument about documentation over and over again. This has also helped us a lot at Google, because programmers can spend endless amount of time arguing about comments, placement of curly braces, etc. and having a written policy helps cut down on that. To give an idea of how to write good comments, here are two examples of undocumented code I ran into in GHC and how better comments would have helped. *First example* In compiler/nativeGen/X86/Instr.hs there's a (local) function called mkRUR, which is a helper function use when computing instruction register usage. The first question that I asked upon seeing uses of that function was "what does RUR stand for?" Given the context the function is in, I guessed it stands for read-update-read, because R is used to mean "read" in the enclosing function and "updating" is related to "reading" so that must be what U stands for. It turns out that it stands for RegUsageReadonly. Here's a comment that would have captured, in a single sentence, what this function is for: -- | Create register usage info for instruction that only -- reads registers. mkRUR src = src' `seq` RU src' [] where src' = filter (interesting platform) src That already a big improvement. A note about the register filtering, which means that not all registers you pass to the function will be recorded as being read in the end, could also be useful. Aside: providing a type signature, which would have made it clear that the return type is RU, might also have helped in this particular case. *Second example* In the same file there a function called x86_regUsageOfInstr. It's the function that encloses the local function mkRUR above. I could figure out that this function has something to do with register usage, of the instruction passed as an argument, and that register usage is important for the register allocator. However, trying to understand in more detail what that meant was more of challenge than it needed to be. First, a comment more clearly explaining what computing register usage means in practice would be helpful: -- | Returns which registers are read and written by this -- instruction, as a (read, written) pair. This info is used -- by the register allocator. x86_regUsageOfInstr :: Platform -> Instr -> RegUsage The reason mentioning that the return value is essentially a (read, written) pair is helpful is because the body of the function a big case statement full of lines like this one: GCMP _ src1 src2 -> mkRUR [src1,src2] ... FDIV _ src dst -> usageRM src dst It's not immediately clear that all the various helper functions used here just end up computing a pair of the above form. A top-level comment lets you understand what's going on without understanding exactly what all these helper functions are doing. Thoughts? -- Johan

15 23

Re: [GHC] #9304: Floating point woes; Different behavior on Mac vs Linux
by Levent Erkok 13 Jul '14

13 Jul '14

So, it appears that the one ending with 21 is the likely correct result; as opposed to 22. Is this an issue with some underlying library (glibc etc.); or an issue with GHC itself? On Sat, Jul 12, 2014 at 8:03 PM, GHC <ghc-devs(a)haskell.org> wrote: > #9304: Floating point woes; Different behavior on Mac vs Linux > -------------------------------------+------------------------------------ > Reporter: lerkok | Owner: > Type: bug | Status: new > Priority: high | Milestone: > Component: Compiler | Version: 7.8.3 > Resolution: | Keywords: floating point > Operating System: Unknown/Multiple | Architecture: Unknown/Multiple > Type of failure: None/Unknown | Difficulty: Unknown > Test Case: | Blocked By: 9276 > Blocking: | Related Tickets: > -------------------------------------+------------------------------------ > > Comment (by carter): > > I ran it on a 64 bit linux server i have using ghci > {{{ > Prelude> :set -XScopedTypeVariables > Prelude> let x :: Double = -4.4 > Prelude> let y :: Double = 2.4999999999999956 > Prelude> decodeFloat (x*y) > (-6192449487634421,-49) > > }}} > > so if anything, it looks like its 32bit vs 64bit > > could you try running the above snippet in GHCi on your 32bit machine? > > -- > Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9304#comment:4> > GHC <http://www.haskell.org/ghc/> > The Glasgow Haskell Compiler >

1 0

Note: Phabricator can now build code reviews!
by Austin Seipp 12 Jul '14

12 Jul '14

Hi *, Quick notice: I spent some time hacking to get Phabricator to build code reviews you publish against the GHC repository. It luckily was not that hard to do so - hooray! Go here for an example: https://phabricator.haskell.org/D64 Look at the top set of information, and find 'Build Status'. This points to the current build being produced for this revision. Icon colors are - red is failed, blue is in progress, green means good. The build is driven by an application called 'Harbormaster'. Harbormaster makes an HTTP request when a new diff is posted, which triggers a build on a backend system that applies the diff and runs ./validate. Now, look at this specific link: https://phabricator.haskell.org/D64#16 I updated the diff, and the build failed and notified Harbormaster. This adds a notification to the page (just below the comment I linked), emails interested parties, and marks the build status at the top. After a build happens, a bot named `phaskell` will drop by and give some results. Validate logs are gzip'd, and uploaded to Phabricator as .txt.gz files. You can download them and look at the full results. If the build fails, the tests that failed are output in the comment on the revision automatically. Builds happen per diff. This means they happen every time you run 'arc diff'. This is all extremely new. Please be nice to it. :) However - if you have some cool changes, please post them! I might just post some to let it do ./validate - it's nice if the machines can just do the work and you can be lazy. :) In the future, Phabricator's Drydock and Harbormaster applications will expand to hopefully do this on their own, with no custom infrastructure from us. We can also utilize resources more effectively. Here's where you can look for the current build queue: https://phabricator.haskell.org/harbormaster/query/all/ The build queue is currently serial - that means the system only runs one build at a time on a 4 core dedicated machine. This simplifies the management but it does mean it could get slightly backed up. It's only a 30 minute wait to validate something though, and that can probably be improved further. Next up: Linking Phabricator to Trac. P.S. The way it works technically is Phabricator -> HTTP GET -> Build machine. This machine runs an HTTP Server - a Python Flask server that simply runs a build script, which reports back to phabricator.haskell.org with JSON requests. Phabricator requests are HTTPS secured. GET requests are securely tunneled over an encrypted pipe (spiped). This build script update Harbormaster's status, post comments, and uploads the build log. Source code: https://github.com/haskell-infra/phab-ghc-builder -- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

1 0

Status of Haskell Platform 2014.2.0.0
by Mark Lentczner 11 Jul '14

11 Jul '14

The status is: *Good-to-Go!* The new-build branch of Haskell Platform is in pretty great shape: - One consistent build system using Shake - Builds source tarball - Builds linux distribution tarball - Builds Mac installer - Builds in one command line from a GHC bindist to end-user installer! - We have a running travis-ci instance <https://travis-ci.org/haskell/haskell-platform>* (currently red as we await a 7.8.3 bindist)* As soon as GHC 7.8.3 is out, I'll be running an alpha (or is this rc1?) or Haskell Platform (well, after I build the bindists for GHC for Mac... so it'll be a few hours on my puny MacBook Air...) *Shout out to: Yitzchak Gale, Bob Ipoloito, and Randy Polen for lots of help with the new build; Carter Schonwald for Mac and GHC build issues; Robert Lefkowitz for wiki and issue converstion (now on github!); Neil Mitchell for consultations on Shake.* The platform is now in a place that we'll be able to turn it much more quickly. This means we can track GHC release more closely, handle important library fixes when needed. And more importantly, spend time improving the platform itself, rather than sapping our energies build it! What you need to do now: - Check the version list in Release2014.hs <https://github.com/haskell/haskell-platform/blob/new-build/hptool/src/Relea…> - If you see a problem with the version of your library, let us know - If you are a packager for one of the linux distros - please try out the build and see how it works, and how it meshes with your packaging process for the distro. Come talk to me if you need help or brainstorming w.r.t. to the new-build. - If you have a Mac, you can try building the platform. *Build note: The repo is in sync with head of GHC 7.8 and we've been working with bindists we build ourselves off of head. If you want to try out the HP build with a 7.8.2 bindist, you'll need to change the version of base back to 4.7.0.0.* What you can do to help There are few aspects in progress, but none of these are show stoppers: - Windows installer using the new build being written by Randy Polen. (Existing windows build from the source tarball is still possible) - contact him if you'd like to help. - Platform website is being revamped by Erin Depew in concert with the general Haskell redesign by Chris Done - Platform website is template generated... but the pages haven't been templated much. if you're interested, e-mail and I can supply more details and hook up interested parties and Erin. - The travis-ci instance is enabled for Mac builds, but isn't configured to do them right. - There is a TO DO small task list in BUILD-NEW <https://github.com/haskell/haskell-platform/blob/new-build/notes/BUILD-NEW#…> in the repo - I'd like to see the As and Bs done. If you want to tackle one just do it and send me a merge request. - Many people have asked for a "server edition" of the platform w/o packages that don't make sense on a server (such as the OpenGL stuff). The new-build could easily be extended to do this now. - I'd love to see a version of Simon Hengel's Haskell Platform Versions Comparison Chart <http://sol.github.io/haskell-platform-versions-comparison-chart/> as part of the platform website. Now that it is templatized and the raw data is in Haskell, it should be possible. (I have prior year's data in the right format, just not checked in, ask me...) - It would be wonderful if we could get tests incorporated: Both running the tests that are part of the included libraries, and perhaps some "big integration tests" (is compiling Pandoc enough?). It would be great if these tests can be run "in-place" after the platform is built (for travis-ci), and if they could be run on the target machine (post-installation). — Mark "is it July already?" Lentczner

5 4

Re: [commit: packages/base] ghc-7.8: Update changelog.md (d762c0c)
by Gabor Greif 11 Jul '14

11 Jul '14

See this comment on Reddit: http://www.reddit.com/r/haskell/comments/2af4jf/ghc783_is_out/ciufcfc Gabor On 7/11/14, git(a)git.haskell.org <git(a)git.haskell.org> wrote: > Repository : ssh://git@git.haskell.org/base > > On branch : ghc-7.8 > Link : > http://ghc.haskell.org/trac/ghc/changeset/d762c0c11b3e9b3031c76caaa895202d1… > >>--------------------------------------------------------------- > > commit d762c0c11b3e9b3031c76caaa895202d1b81acdf > Author: Herbert Valerio Riedel <hvr(a)gnu.org> > Date: Fri Jul 11 13:58:41 2014 +0200 > > Update changelog.md > > >>--------------------------------------------------------------- > > d762c0c11b3e9b3031c76caaa895202d1b81acdf > changelog.md | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > diff --git a/changelog.md b/changelog.md > index 5eef01f..bb42c1e 100644 > --- a/changelog.md > +++ b/changelog.md > @@ -1,6 +1,6 @@ > # Changelog for [`base` package](http://hackage.haskell.org/package/base) > > -## 4.7.0.1 *Jun 2014* > +## 4.7.0.1 *Jul 2014* > > * Bundled with GHC 7.8.3 > > @@ -11,6 +11,8 @@ > > * Fix regression in Data.Fixed Read instance (#9231) > > + * Fix `fdReady` to honor `FD_SETSIZE` (#9168) > + > ## 4.7.0.0 *Mar 2014* > > * Bundled with GHC 7.8.1 > > _______________________________________________ > ghc-commits mailing list > ghc-commits(a)haskell.org > http://www.haskell.org/mailman/listinfo/ghc-commits >

1 0

Explicitly bidirectional pattern synonyms for 7.8.3?
by Dr. ERDI Gergo 11 Jul '14

11 Jul '14

Hi, I've spent the last couple evenings implementing explicitly-bidirectional pattern synonyms, partly while waiting for someone to help out with #9023. Explicitly-bidirectional pattern synonyms are described in #8581. My question is: now that I have this mostly working, is this something that could be added to 7.8.3, or is this too much of a new feature for a point release? I'd like to know because then I can prioritize finishing the work (it still needs some tests added and also someone to review the changes). Thanks, Gergo

3 3

Injective type families
by Jan Stolarek 11 Jul '14

11 Jul '14

Hi all, I'd like to take a stab at implementing injective type families (#6018). My plan of attack looks like this: 1. Implement injective type families that are: a) injective in all arguments b) only admit RHS that is a concrete type or a type variable introduced by the LHS or a recursive call to self. 2. Lift restriction a) ie. allow type families injective only in some arguments 3. Lift restriction b) ie. allow injective type families to call other type families. I'm not sure if 3) really gets implemented as this seems more in lines of #4259, which I don't intend to approach. Now, let's discuss the most important of all matters - the syntax! :-) Note that syntax must: 1. allow to define injectivity only for some parameters 2. work for both open and closed type families Here's my proposal (based on Richard's idea to use functional-dependencies-like syntax): 1. Standard injective type family (all parameters uniquely determined by the RHS): injective type family F a b c | a b c type family instance F Int Char Bool = Bool type family instance F Char Bool Int = Int type family instance F Bool Int Char = Char 2. Type family injective only in some parameters (ie. only some parameters uniquely determined by the RHS): injective type family G a b c | a b type family instance G Int Char Bool = Bool type family instance G Int Char Int = Bool type family instance G Bool Int Int = Int Here knowing the RHS allows us to determine a and b, but not c. 3. Type families where knowing the RHS and some parameters on the LHS makes other parameters injective: Example 1: knowing the RHS and any single parameter uniquely determines other parameters injective type family H a b c | a -> b c, b -> a c, c -> a b type family instance H Int Char Double = Int type family instance H Bool Double Char = Int Example 2: knowing the RHS and either a or b allows to uniquely determine other parameters, but knowing the RHS and c gives us no information about a or b injective type family J a b c | a -> b c, b -> a c type family instance J Int Char Double = Int type family instance J Bool Double Double = Int For closed type families this notation would be identical: type family declaration would be followed by the "where" keyword. In comment 34 of #6018 Richard proposed a notation that uses a keyword "Result". This would allow for a more uniform notation. Notice that in my proposal some families use ->, while others don't. The idea is that on the right side of the arrow we write what we can infer from the things on the left of the arrow. If the only thing left of the arrow is the right hand side of the type family equation (the result) then the arrow can be elided. In other words these would be equivalent: injective type family F a b c | a b c injective type family F a b c | result -> a b c injective type family G a b c | a b injective type family G a b c | result -> a b (note: it is also true that "injective type family G a b c | result c -> a b") injective type family H a b c | a -> b c, b -> a c, c -> a b injective type family H a b c | result a -> b c, result b -> a c, result c -> a b injective type family J a b c | a -> b c, b -> a c injective type family J a b c | result a -> b c, result b -> a c I find the notation explicitly using "result" to be easier to understand, but it is also more verbose. I also suspect that it might be a bit easier to parse. Which of the notations do you find better? What do you think about using the "injective" keyword? Are there any alternative proposals? Janek

6 7

GHC silently turns off dynamic output, should this be an error?
by Christopher Rodrigues 08 Jul '14

08 Jul '14

Hello, I’ve investigated an issue (#9176) where ghc with --make -dynamic-too appears to run successfully but doesn’t generate some .dyn_hi files. The cause seems to be that GHC silently turns off dynamic-too output when it finds an inconsistency between the .hi and .dyn_hi interface hashes of an imported module. The symptom, for me, is that some cabal packages will build but fail to install. The real problem is most likely not in the failing package, but in an imported module from a package that’s already installed. I’m inclined to say that GHC should stop with an error in this situation, which would produce a visible error when the file fails to compile. However, since GHC carefully avoids bailing out, I wonder if I’m missing something. Is there a good reason to turn off dynamic-too output instead of reporting an error? Additionally, is it ever valid to have a pair of .hi and .dyn_hi files with different interface hashes? Or is it invalid data, in the same way that a corrupted .hi file is invalid data? I’m inclined to say that it is invalid data, because -dynamic-too promises to generate two different (static and dynamic) implementations of the same Haskell code, taking two different implementations of the same imported Haskell modules as input. If the inputs aren’t really two different implementations of the same Haskell code, you’ve somehow provided invalid input.

5 5

Continuous Integration and Cross Compilation
by William Knop 08 Jul '14

08 Jul '14

Hello all, I’ve seen quite a few comments on the list and elsewhere lamenting the time it takes to compile and validate ghc. It’s troublesome not only because it’s inconvenient, but, more seriously, people are holding off on sending patches in which stifles development. I would like to propose a solution: 1. Implement proper cross-compilation, such that build and host may be different— e.g. a linux x86_64 machine can build ghc that runs on Windows x86. What sort of work would this entail? 2. Batch cross-compiled builds for all OSs/archs on a continuous integration service (e.g. Travis CI) or cloud service, then package up the binaries with the test suite. 3. Send the package to our buildbots, and run the test suite. 4. (optional) If using a CI service, have the buildbots send results back to the CI. This could be useful if we'd use GitHub for pulls in the future *. Cheers, Will * I realize vanilla GitHub currently has certain annoying limitations, though some of them are pretty easy to solve via the github-services and/or webhooks. I don’t think this conflicts with the desire to use Phabricator, either, so I’ll send details and motivations to that thread.

7 12