- ghc-devs - Haskell.org

Feedback request regarding HSOC project (bringing sanity to the GHC performance test-suite)
by Jared Weakly 15 Sep '17

15 Sep '17

Hey y'all, A quick ToC before I dive right in: * What my HSOC project is on * My progress so far * Feedback welcome * What I have left to do * Theoretical potential improvements ----------- My HSOC project was on bringing sanity to the GHC performance test-suite. My blog post on this is here: https://jaredweakly.com/blog/haskell-summer-of-code/ The Trac ticket that corresponds to this is here: https://ghc.haskell.org/trac/ghc/ticket/12758 The Phabricator ticket for this patch: https://phabricator.haskell.org/D3758 The tl;dr of my HSOC project is that GHC's performance tests currently require the programmer to add in expected numbers manually, updated them, handhold the testsuite, etc. This is a bit absurd and my project's overall aim is to reduce the effort required of the programmer to as close to zero as possible while simultaneously increasing the potential ability of the testsuite to catch regressions as much as possible. ------------ My progress so far: - I have a few comparison tools in perf_notes.py. These allow people to compare performance numbers of tests across commits - I have all the performance numbers generated by running the tests automatically stored in git notes and referenced by both the comparison tool and the testsuite - I have refactored the testsuite to use my new code that pulls expected numbers automatically from git notes (trivially passing if the note does not yet exist for that test), then it compares that expected number with the number that was gotten from running the testsuite on the latest commit. The comparison passes if it's within a certain deviation (20% by default, but can be customized by the programmer). - I have refactored all of the all.T files to use the new comparison functions for the performance tests and ensured that this doesn't break any existing tests. ------------ Anyone who wants to checkout the wip/perf-testsuite and try this out is more than welcome. Feedback on anything is welcome; comments are appreciated; discussion is welcome, etc. ------------- What I have left to do is: 1. Finish writing up the documentation 2. Update the wiki in all the relevant places concerning additions/modifications to the testsuite and test driver 3. Make sure everyone is happy with the change (and make small changes as necessary) -------------- Possible features and improvements I am thinking about adding in: * As a stopgap to full integration with performance tracking tools (such as Gipedia), optionally emitting a test warning with the test summary if there is any regression detected whatsoever (even if the number falls within the allowed deviation) * Some tests, such as T7702, have a somewhat nonsensical regression percentage. Ideally the testsuite could handle those better. I could potentially build in multiple ways to determine a regression (percentage, 'above a certain value', 'taking longer than X amount of time', as potential examples) * Currently some tests require installing some Haskell packages; they are skipped if the packages are not installed. I could try to build in a way to automatically attempt to install all necessary Haskell packages if someone attempts to run a test that requires them. (Perhaps using a command such as 'make test exhaustive') * The performance metric 'peak_megabytes' is sometimes not accurate enough; I could see if adding something like `RTS -h -i0.01` automatically to tests that use 'peak_megabytes' would resolve that. Currently it is a manual debugging step. Any thoughts? Comments? Questions? Regards, Jared Weakly

4 6

Overapproximation of loopbreakers due to unfoldings
by Matthew Pickering 15 Sep '17

15 Sep '17

I have been puzzling over the example presented by Harendra in #14211 for the last two days. Ultimately it led me to discover that if a self-recursive definition is marked with an INLINE pragma then it will always be marked as a loopbreaker. This is undesirable in this case as the simplifier ends up with in the first pass.. bindWith = ... bindWith ... => bindWith = bindWith_abc |> co bindWith_abc = ... bindWith ... So we can then inline `bindWith` into the RHS of `bindWith_abc` and create a single self-recursive function rather than a mutually recursive block. Marking `bindWith` as `INLINE` means that both `bindWith` and `bindWith_abc` are marked as loopbreakers. Mutual recursive blocks are bad as they completely stop the static argument transformation from working. My question is, why is it necessary to mark "bindWith" as a loopbreaker in the current module? Are there any tickets or notes which discuss this problem? Matt

2 1

Phab: conditional approval
by Richard Eisenberg 14 Sep '17

14 Sep '17

Hi devs, When reviewing a diff on Phab, I can "accept" or "request changes". Sometimes, though, I want to do both: I suggest very minor (e.g., typo) changes, but then when these changes are made, I accept. I'm leery of making the suggestions and saying "accept", because then someone working quickly may merge without noticing the typos. Does Phab have such an option? Thanks, Richard

4 5

Perf improvement
by Simon Peyton Jones 14 Sep '17

14 Sep '17

I'm seeing this in validate bytes allocated value is too low: (If this is because you have improved GHC, please update the test so that GHC doesn't regress again) Expected T5837(normal) bytes allocated: 56782344 +/-7% Lower bound T5837(normal) bytes allocated: 52807579 Upper bound T5837(normal) bytes allocated: 60757109 Actual T5837(normal) bytes allocated: 52424864 Deviation T5837(normal) bytes allocated: -7.7 % *** unexpected stat test failure for T5837(normal) Does anyone else? Simon

2 3

./validate --slow results
by Bartosz Nitka 14 Sep '17

14 Sep '17

Hi all, I happened to run ./validate --slow on my linux machine and I thought it would be useful to share the results. Results: Unexpected results from: TEST="EtaExpandLevPoly PatternSplice StrictPats T10508_api T11627b T12809 T12870a T12870b T12870c T12870d T12870e T12870f T12870g T12870h T12903 T12962 T13366 T13543 T13688 T13780c T13822 T13949 T14137 T2552 T2783 T3001-2 T4114c T4114d T4188 T4334 T5129 T5363 T5559 T5611 T680 T7411 T7837 T7944 T8025 T8089 T8542 TH_spliceE5_prof_ext UnsafeReenter compact_gc compact_share dsrun014 dynamic-paper haddock.Cabal hpc_fork prof-doc-fib prof-doc-last profinline001 read029 return_mem_to_os rn041 scc001 scc002 scc003 scc005 space_leak_001 tc165 tryReadMVar2" SUMMARY for test run started at Thu Sep 14 04:57:20 2017 PDT 0:12:27 spent to go through 6091 total tests, which gave rise to 24105 test cases, of which 4531 were skipped 143 had missing libraries 19120 expected passes 203 expected failures 0 caused framework failures 0 caused framework warnings 6 unexpected passes 97 unexpected failures 5 unexpected stat failures Unexpected passes: /tmp/ghctest-e17wqi8b/test spaces/./dependent/should_compile/dynamic-paper.run dynamic-paper [unexpected] (optasm) /tmp/ghctest-e17wqi8b/test spaces/./indexed-types/should_compile/T7837.run T7837 [unexpected] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./parser/should_compile/read029.run read029 [unexpected] (optasm) /tmp/ghctest-e17wqi8b/test spaces/./rename/should_compile/rn041.run rn041 [unexpected] (optasm) /tmp/ghctest-e17wqi8b/test spaces/./simplCore/should_fail/T7411.run T7411 [unexpected] (hpc) /tmp/ghctest-e17wqi8b/test spaces/./typecheck/should_compile/tc165.run tc165 [unexpected] (optasm) Unexpected failures: /tmp/ghctest-e17wqi8b/test spaces/./codeGen/should_run/T5129.run T5129 [bad exit code] (hpc) /tmp/ghctest-e17wqi8b/test spaces/./codeGen/should_run/T5129.run T5129 [bad exit code] (optasm) /tmp/ghctest-e17wqi8b/test spaces/./codeGen/should_run/T5129.run T5129 [bad exit code] (threaded2) /tmp/ghctest-e17wqi8b/test spaces/./codeGen/should_run/T5129.run T5129 [bad exit code] (dyn) /tmp/ghctest-e17wqi8b/test spaces/./concurrent/should_run/T5611.run T5611 [bad stdout] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./concurrent/should_run/tryReadMVar2.run tryReadMVar2 [bad heap profile] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./deSugar/should_run/dsrun014.run dsrun014 [bad stderr] (hpc) /tmp/ghctest-e17wqi8b/test spaces/./deSugar/should_run/dsrun014.run dsrun014 [bad stderr] (optasm) /tmp/ghctest-e17wqi8b/test spaces/./deSugar/should_run/dsrun014.run dsrun014 [bad stderr] (threaded2) /tmp/ghctest-e17wqi8b/test spaces/./deSugar/should_run/dsrun014.run dsrun014 [bad stderr] (dyn) /tmp/ghctest-e17wqi8b/test spaces/./dependent/should_fail/T13780c.run T13780c [stderr mismatch] (normal) /tmp/ghctest-e17wqi8b/test spaces/./dependent/should_compile/dynamic-paper.run dynamic-paper [exit code non-0] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./driver/T4114d.run T4114d [bad exit code] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./driver/T4114c.run T4114c [bad exit code] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./ghc-api/T10508_api.run T10508_api [bad exit code] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./numeric/should_compile/T8542.run T8542 [stderr mismatch] (hpc) /tmp/ghctest-e17wqi8b/test spaces/./ghc-api/T10508_api.run T10508_api [bad exit code] (profthreaded) /tmp/ghctest-e17wqi8b/test spaces/./partial-sigs/should_compile/PatternSplice.run PatternSplice [exit code non-0] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./patsyn/should_run/T13688.run T13688 [exit code non-0] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./patsyn/should_run/T13688.run T13688 [exit code non-0] (profthreaded) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T3001-2.run T3001-2 [exit code non-0] (prof_hb) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/scc003.run scc003 [bad profile] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T11627b.run T11627b [bad heap profile] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T12962.run T12962 [bad profile] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/scc003.run scc003 [bad profile] (profthreaded) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T12962.run T12962 [bad profile] (profthreaded) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/scc003.run scc003 [bad profile] (prof) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T12962.run T12962 [bad profile] (prof) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/scc001.run scc001 [bad exit code] (ghci-ext-prof) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/scc005.run scc005 [bad exit code] (ghci-ext-prof) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/profinline001.run profinline001 [bad exit code] (ghci-ext-prof) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T5559.run T5559 [bad exit code] (ghci-ext-prof) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T11627b.run T11627b [bad heap profile] (prof_hc_hb) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/scc003.run scc003 [bad exit code] (ghci-ext-prof) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T680.run T680 [bad exit code] (ghci-ext-prof) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/scc002.run scc002 [bad exit code] (ghci-ext-prof) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T12962.run T12962 [bad exit code] (ghci-ext-prof) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/prof-doc-fib.run prof-doc-fib [bad exit code] (ghci-ext-prof) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T11627b.run T11627b [bad heap profile] (prof_hb) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T2552.run T2552 [bad exit code] (ghci-ext-prof) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/prof-doc-last.run prof-doc-last [bad exit code] (ghci-ext-prof) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T11627b.run T11627b [bad heap profile] (prof_hd) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T5363.run T5363 [bad exit code] (ghci-ext-prof) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T11627b.run T11627b [bad heap profile] (prof_hy) /tmp/ghctest-e17wqi8b/test spaces/./profiling/should_run/T11627b.run T11627b [bad heap profile] (prof_hr) /tmp/ghctest-e17wqi8b/test spaces/./rts/T2783.run T2783 [bad exit code] (threaded1) /tmp/ghctest-e17wqi8b/test spaces/./rts/return_mem_to_os.run return_mem_to_os [bad heap profile] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./rts/T12903.run T12903 [bad heap profile] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870a.run T12870a [bad stderr] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870f.run T12870f [bad stdout] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870b.run T12870b [bad heap profile] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870c.run T12870c [bad heap profile] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870e.run T12870e [bad stdout] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870d.run T12870d [bad stderr] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870b.run T12870b [bad exit code] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870f.run T12870f [bad stderr] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870c.run T12870c [bad exit code] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870e.run T12870e [bad stderr] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870g.run T12870g [bad stderr] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870h.run T12870h [bad heap profile] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870h.run T12870h [bad stderr] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870f.run T12870f [bad stdout] (threaded2) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870e.run T12870e [bad stdout] (threaded2) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870f.run T12870f [bad stdout] (profthreaded) /tmp/ghctest-e17wqi8b/test spaces/./rts/flags/T12870e.run T12870e [bad stdout] (profthreaded) /tmp/ghctest-e17wqi8b/test spaces/./simplCore/should_compile/T7944.run T7944 [exit code non-0] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./simplCore/should_compile/T13543.run T13543 [stderr mismatch] (hpc) /tmp/ghctest-e17wqi8b/test spaces/./simplCore/should_compile/T13543.run T13543 [stderr mismatch] (optasm) /tmp/ghctest-e17wqi8b/test spaces/./simplCore/should_compile/T13543.run T13543 [stderr mismatch] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./simplCore/should_compile/T14137.run T14137 [stderr mismatch] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./th/TH_spliceE5_prof_ext.run TH_spliceE5_prof_ext [bad exit code] (normal) /tmp/ghctest-e17wqi8b/test spaces/./th/T4188.run T4188 [exit code non-0] (normal) /tmp/ghctest-e17wqi8b/test spaces/./th/T4188.run T4188 [exit code non-0] (ext-interp) /tmp/ghctest-e17wqi8b/test spaces/./th/T13366.run T13366 [bad stderr] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./th/should_compile/T8025/T8025.run T8025 [exit code non-0] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./th/should_compile/T13949/T13949.run T13949 [exit code non-0] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./typecheck/should_compile/T13822.run T13822 [exit code non-0] (normal) /tmp/ghctest-e17wqi8b/test spaces/./typecheck/should_compile/T13822.run T13822 [exit code non-0] (hpc) /tmp/ghctest-e17wqi8b/test spaces/./typecheck/should_compile/T13822.run T13822 [exit code non-0] (optasm) /tmp/ghctest-e17wqi8b/test spaces/./typecheck/should_compile/T13822.run T13822 [exit code non-0] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./typecheck/should_run/EtaExpandLevPoly.run EtaExpandLevPoly [exit code non-0] (profasm) /tmp/ghctest-e17wqi8b/test spaces/./typecheck/should_run/EtaExpandLevPoly.run EtaExpandLevPoly [bad stderr] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./typecheck/should_run/T12809.run T12809 [bad stderr] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./typecheck/should_run/StrictPats.run StrictPats [bad stderr] (ghci) /tmp/ghctest-e17wqi8b/test spaces/./typecheck/should_run/EtaExpandLevPoly.run EtaExpandLevPoly [exit code non-0] (profthreaded) /tmp/ghctest-e17wqi8b/test spaces/../../libraries/base/tests/T8089.run T8089 [bad exit code] (ghci) /tmp/ghctest-e17wqi8b/test spaces/../../libraries/base/tests/T8089.run T8089 [bad exit code] (threaded1) /tmp/ghctest-e17wqi8b/test spaces/../../libraries/base/tests/T8089.run T8089 [bad exit code] (threaded2) /tmp/ghctest-e17wqi8b/test spaces/../../libraries/base/tests/T8089.run T8089 [bad exit code] (profthreaded) /tmp/ghctest-e17wqi8b/test spaces/../../libraries/hpc/tests/fork/hpc_fork.run hpc_fork [bad heap profile] (profasm) /tmp/ghctest-e17wqi8b/test spaces/../../libraries/ghc-compact/tests/compact_share.run compact_share [bad stdout] (profasm) /tmp/ghctest-e17wqi8b/test spaces/../../libraries/ghc-compact/tests/compact_share.run compact_share [bad stdout] (ghci) /tmp/ghctest-e17wqi8b/test spaces/../../libraries/ghc-compact/tests/compact_gc.run compact_gc [bad heap profile] (profasm) /tmp/ghctest-e17wqi8b/test spaces/../../libraries/ghc-compact/tests/compact_share.run compact_share [bad stdout] (profthreaded) /tmp/ghctest-e17wqi8b/test spaces/./ffi/should_fail/UnsafeReenter.run UnsafeReenter [bad exit code] (threaded1) /tmp/ghctest-e17wqi8b/test spaces/./ffi/should_fail/UnsafeReenter.run UnsafeReenter [bad exit code] (threaded2) /tmp/ghctest-e17wqi8b/test spaces/./ffi/should_fail/UnsafeReenter.run UnsafeReenter [bad exit code] (profthreaded) Unexpected stat failures: /tmp/ghctest-e17wqi8b/test spaces/./perf/haddock/haddock.Cabal.run haddock.Cabal [stat not good enough] (normal) /tmp/ghctest-e17wqi8b/test spaces/./perf/space_leaks/space_leak_001.run space_leak_001 [stat too good] (hpc) /tmp/ghctest-e17wqi8b/test spaces/./perf/space_leaks/space_leak_001.run space_leak_001 [stat too good] (optasm) /tmp/ghctest-e17wqi8b/test spaces/./perf/space_leaks/T4334.run T4334 [stat not good enough] (threaded2) /tmp/ghctest-e17wqi8b/test spaces/./perf/space_leaks/space_leak_001.run space_leak_001 [stat too good] (dyn) >From my point of view, with this amount of failures it's difficult to tell if a new change introduces problems. Shall we mark them as broken and open tickets for follow ups? Cheers, Bartosz

3 2

nofib behind master?
by Joachim Breitner 13 Sep '17

13 Sep '17

Hi, nofib currently does not build (and hence perf.haskell.org is broken), du to the Monoid-Semigroup-Change. I looked into it and found that nofib, as refernced by ghc HEAD, is 26 commits behind nofib master. Ben, you did some of those commits. Any good reason not to update the submodule in ghc? Greetings, Joachim -- Joachim “nomeata” Breitner mail(a)joachim-breitner.de https://www.joachim-breitner.de/

2 1

GHC 8.2.1 release commit
by Harendra Kumar 13 Sep '17

13 Sep '17

Hi, In the GHC git repo how do I figure out which commit belongs to release 8.2.1. I cannot find 8.2.1 in "git tag" output. I tried "git log" and searching for 8.2.1 but there seems to be no definitive comment marking 8.2.1. -harendra

2 2

Does span run in bounded space?
by Facundo Domínguez 13 Sep '17

13 Sep '17

Dear devs, I used to think that span is a function which doesn't run in bounded space. But ghc defied this understanding. The program copied below runs in bounded space with "ghc-8.2.1 -O0 main.hs; ./main" but it does not if run with "runghc-8.2.1 -O0 main.hs". Does someone have any insights on what's the optimization that is making a difference in the memory footprint of these two methods of running? Or is there a bug in ghci, and span is supposed to run in bounded space after all? Thanks, Facundo import Prelude hiding (span) import Data.List (foldl') main = case span (>0) (replicate 100000000 1) of (xs, ys) -> do print (foldl' (+) 0 xs) print (foldl' (+) 0 ys) span :: (a -> Bool) -> [a] -> ([a],[a]) span _ xs@[] = (xs, xs) span p xs@(x:xs') | p x = let (ys, zs) = span p xs' in (x : ys, zs) | otherwise = ([],xs)

1 0

Feedback on applying implicit exceptions wanted
by Andreas Klebinger 13 Sep '17

13 Sep '17

I'm currently working out ideas for potential optimizations in pattern matching compilation for GHC. While I think the theory works out fine in regards to exceptions I would like feedback on the possible implications. Trac ticket with details is here: https://ghc.haskell.org/trac/ghc/ticket/14201#comment:4 As for the theory I worked based of the assumptions that error = bottom and pattern match failure = error. Otherwise we get different results from both cases even in theory. While I mostly worry about the practical implications I would also appreciate any other feedback.

1 0

Semigroup repeat (base package)
by Harendra Kumar 12 Sep '17

12 Sep '17

Hi, I am sending this question here since base ships with ghc, let me know if this is not the right forum for this. I could not find a function that repeats a value using a semigroup append. I am looking for something like this: srepeat :: Semigroup a => a -> a srepeat x = xs where xs = x <> xs Is it already available somewhere? Does it make sense to add it to Data.Semigroup? Thanks, Harendra

4 7