September 2015 - Glasgow-haskell-users

Discovery of source dependencies without --make
by Lars Hupel 13 Dec '15

13 Dec '15

(cross-posted from [haskell-cafe]) Hi, I've got a problem in our – admittedly complex – build process. We're running an automated grading system for student-submitted homework exercises. The compilation proceeds in stages: 1) Compile a library file, marked as trustworthy 2) Compile student submissions using Safe Haskell, able to access a whitelisted set of packages and the previously compiled library files 3) Compile the test suite 4) Link everything together into an executable The invocation is similar to this: ghc -c -outputdir "$OUT" -XTrustworthy Library.hs ghc -c -outputdir "$OUT" -i"$OUT" -XSafe "$SUBMISSION" ghc -c -outputdir "$OUT" -i"$OUT" Test_Suite.hs ghc -outputdir "$OUT" -i"$OUT" -o "$OUT/runner" The second stage works well under the assumption that there's just one single submitted file. If there's more than one, I need to specify them in topological order wrt their module dependencies. Consider two simple modules: A.hs > module A where > > import B > import Library B.hs > module B where Invoking GHC with "A.hs" "B.hs" will fail: A.hs:3:1: Failed to load interface for ‘B’ My naive attempt at solving that problem was to just insert "--make" into the compiler flags. No luck, because according to [1, §4.7.3], "--make" will only look at source files and will ignore the "Library.hi" file from the previous compilation stage. If I give the path to "Library.hs" as well, GHC insists on recompiling it with changed flags ("-XSafe" instead of "-XTrustworthy"). Reading a bit further, I discovered the '-M' flag, however, not only does it output Makefile-formatted output, it also ignores .hi files. Is there any way to get the dependency discovery of '--make' without the rest? I know I could probably make it work if I made a package out of the first build stage, but I really want to avoid that in order to not increase the build complexity even further. Cheers Lars [1] <https://downloads.haskell.org/~ghc/7.6.3/docs/html/users_guide/separate-com…>

8 15

Removing latency spikes. Garbage collector related?
by Will Sewell 01 Oct '15

01 Oct '15

Hi, I was told in the #haskell IRC channel that this would be a good place to ask this question, so here goes! We’re writing a low-latency messaging system. The problem is we are getting a lot of latency spikes. See this image: http://i.imgur.com/GZ0Ek98.png (yellow to red is the 90th percentile), which shows end-to-end latency of messages through the system. I have tried to eliminate the problem by removing parts of the system that I suspected to be expensive, but the spikes are still there. I’m now thinking that it’s the GC. As you can see in this output from ghc-events-analyze, work on the GC thread (red) seems to be blocking work on the main program thread (green) http://i.imgur.com/4YO5q4U.png (x axis is time, darkness of buckets is % CPU time). *Note: the graphs are not of the same run, but are typical* Do you think the GC is the most likely culprit? Is there anything I can do to confirm this hypothesis? (I looked into turning off the GC, but this seems tricky) If it is the GC, then is there anything that can be done about it? Thanks! Will

3 5

Typing pattern synonyms
by Sven Panne 30 Sep '15

30 Sep '15

The type of a pattern synonym like pattern FOO = 1234 seems to be '(Eq a, Num a) => a', which makes partially makes sense, although it's not immediately clear to me where the 'Eq a' part comes from. But probably that would be clear if I read the desugaring rules closely enough. ;-) My real question is about: pattern FOO = 1234 :: Int This doesn't compile out of the box, GHC seems to require ScopedTypeVariables, too: Illegal type signature: `Int' Perhaps you intended to use ScopedTypeVariables In a pattern type-signature Why is this the case? From a user perspective, the error is totally bogus, there are no visible type variables at all. Can GHC be fixed to avoid enabling ScopedTypeVariables? Cheers, S.

3 4

HEADS UP: Need 7.10.3?
by Moritz Drexl 29 Sep '15

29 Sep '15

Hi all, For me, a 7.10.3 release is also urgently needed. Showstopper is the problem regarding invocation of GCC under Windows, with response files as an already implemented solution. See: https://ghc.haskell.org/trac/ghc/ticket/10726 https://ghc.haskell.org/trac/ghc/ticket/10795 https://github.com/ghc/ghc/commit/296bc70b5ff6c853f2782e9ec5aa47a52110345e I am unable to compile a working GHC version of the patched 7.10 branch. I exhausted all other avenues and am unable to make releases for customers using Windows. Thanks! Regards Moritz Drexl

2 1

Fwd: Removing latency spikes. Garbage collector related?
by Will Sewell 29 Sep '15

29 Sep '15

Thanks for the reply Greg. I have already tried tweaking these values a bit, and this is what I found: * I first tried -A256k because the L2 cache is that size (Simon Marlow mentioned this can lead to good performance http://stackoverflow.com/a/3172704/1018290) * I then tried a value of -A2048k because he also said "using a very large young generation size might outweigh the cache benefits". I don't exactly know what he meant by "a very large young generation size", so I guessed at this value. Is it in the right ballpark? * With -H, I tried values of -H8m, -H32m, -H128m, -H512m, -H1024m But all lead to worse performance over the defaults (and -H didn't really have much affect at all). I will try your suggestion of setting -A to the L3 cache size. Are there any other values I should try setting these at? As for your final point, I have run space profiling, and it looks like >90% of the memory is used for our message index, which is a temporary store of messages that have gone through the system. These messages are stored in aligned chunks in memory that are merged together. I initially though this was causing the spikes, but they were still there even after I removed the component. I will try and run space profiling in the build with the message index. Thanks again. On 28 September 2015 at 19:02, Gregory Collins <greg(a)gregorycollins.net> wrote: > > On Mon, Sep 28, 2015 at 9:08 AM, Will Sewell <me(a)willsewell.com> wrote: >> >> If it is the GC, then is there anything that can be done about it? > > Increase value of -A (the default is too small) -- best value for this is L3 > cache size of the chip > Increase value of -H (total heap size) -- this will use more ram but you'll > run GC less often > This will sound flip, but: generate less garbage. Frequency of GC runs is > proportional to the amount of garbage being produced, so if you can lower > mutator allocation rate then you will also increase net productivity. > Built-up thunks can transparently hide a lot of allocation so fire up the > profiler and tighten those up (there's an 80-20 rule here). Reuse output > buffers if you aren't already, etc. > > G > > -- > Gregory Collins <greg(a)gregorycollins.net>

3 5

ANN: CfN for new Haskell Prime language committee
by Herbert Valerio Riedel 23 Sep '15

23 Sep '15

Dear Haskell Community, In short, it's time to assemble a new Haskell Prime language committee. Please refer to the CfN at https://mail.haskell.org/pipermail/haskell-prime/2015-September/003936.html for more details. Cheers, hvr -- PGP fingerprint: 427C B69A AC9D 00F2 A43C AF1C BA3C BA3F FE22 B574

1 0

HEADS UP: Need 7.10.3?
by Austin Seipp 17 Sep '15

17 Sep '15

Hi *, (This is an email primarily aimed at users reading this list and developers who have any interest). As some of you may know, there's currently a 7.10.3 milestone and status page on our wiki: https://ghc.haskell.org/trac/ghc/wiki/Status/GHC-7.10.3 The basic summary is best captured on the above page: "We have not yet decided when, or even whether, to release GHC 7.10.3. We will do so if (but only if!) we have documented cases of "show-stoppers" in 7.10.2. Namely, cases from users where - You are unable to use 7.10.2 because of some bug - There is no reasonable workaround, so you are truly stuck - We know how to fix it - The fix is not too disruptive; i.e. does not risk introducing a raft of new bugs" That is, we're currently not fully sold on the need for a release. However, the milestone and issue page serve as a useful guide, and also make it easier to keep track of smaller, point-release worthy issues. So in the wake of the 8.0 roadmap I just sent: If you *need* 7.10.3 because the 7.10.x series has a major regression or problem you can't work around, let us know! - Find or file a bug in Trac - Make sure it's highest priority - Assign it to the 7.10.3 milestone - Follow up on this email if possible, or edit it on the status page text above - it would be nice to get some public feedback in one place about what everyone needs. Currently we have two bugs on the listed page in the 'show stopper category', possibly the same bug, which is a deal-breaker for HERMIT I believe. Knowing of anything else would be very useful. Thanks all! -- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

3 2

Re: HEADS UP: Need 7.10.3?
by Ruben Moor 16 Sep '15

16 Sep '15

Hello, I want to support the case of a GHC 7.10.3 release. The show-stopper for our company in 7.10.2 consists of ticket #10726 https://ghc.haskell.org/trac/ghc/ticket/10726 and ticket #10795 https://ghc.haskell.org/trac/ghc/ticket/10795 I.e. I am not able to compile my code on MS Windows and reached a point where my workaround is exhausted (reducing overall file path length in the project dependency). Luckily the fix has already been implemented and merged (into the 7.10 branch and HEAD). Cf. this commit https://github.com/ghc/ghc/commit/296bc70b5ff6c853f2782e9ec5aa47a52110345e and this stack issue: https://github.com/commercialhaskell/stack/issues/795#issuecomment-140618684 Thanks! Ruben Moor

1 0

HEADS UP (devs, users): 8.0.1 Roadmap
by Austin Seipp 14 Sep '15

14 Sep '15

Hi *, I've returned from vacation, and last week Simon, Simon and I met up again after a long break, and talked a bit about the upcoming release. The good news is that it is going to be an exciting one! The flip side is, there's a lot of work to be done! The current plan we'd roughly like to follow is... - November: Fork the new `ghc-8.0` STABLE branch - At this point, `master` development will probably slow as we fix bugs. - This gives us 2 months or so until branch, from Today. - This is nice as the branch is close to the first RC. - December: First release candidate - Mid/late February: Final release. Here's our current feature roadmap (in basically the same place as all our previous pages): https://ghc.haskell.org/trac/ghc/wiki/Status/GHC-8.0.1 As time goes on, this page will be updated to reflect Reality™ and track it as closely as possible. So keep an eye on it! It's got the roadmap (near top) and large bug list (bottom). Now, there are some things we need, so depending on who you are, please... - *Users*: please look over the bug list! If there's a bug you need fixed that isn't there, set it to the 8.0.1 milestone (updated in Trac). If this bug is critical to you, please let us know! You can bump the priority (if we disagree, or it's workaround-able, it might get bumped down). We just need a way to see what you need, so please let us know somehow! As a reminder, our regular policy is this: if a bug is NOT marked highest or high priority, it is essentially 100% the case we will not look at it. So please make sure this is accurate. Or if you can, write a patch yourself! - *Developers*: double check the roadmap list, _and if you're responsible for something, make sure it is accurate!_ There are some great things planned to land in HEAD, but we'll have to work for it. Onward! - A better LLVM backend for Tier-1 platforms - Types are kinds and kind equality - Overloaded record fields! - Enhancements to DWARF debugging - ApplicativeDo - ... and many more... Thanks everyone! -- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

1 0