November 2018 - ghc-devs

The future of Phabricator
by Ben Gamari 18 Sep '21

18 Sep '21

TL;DR. For several reasons I think we should consider alternatives to Phabricator. My view is that GitLab seems like the best option. Hello everyone, Over the past year I have been growing increasingly weary of our continued dependence on Phabricator. Without a doubt, its code review interface is the best I have used. However, for a myriad of reasons I am recently questioning whether it is still the best tool for GHC's needs. # The problem There are a number of reasons why I am currently uncertain about Phabricator. For one, at this point we have no options for support in the event that something goes wrong as the company responsible for Phabricator, Phacility, has closed their support channels to non-paying customers. Furthermore, in the past year or two Phacility has been placing their development resources in the parts their customers pay them for, which appear to be much different that the parts that we actively use. For this reason, some parts that we rely on seem oddly half-finished. This concern was recently underscored by some rather unfortunate misfeatures in Harbormaster which resulted in broken CI after the Hadrian merge and now apparent bugs which have made it difficult to migrate to the CircleCI integration we previously prepared. Perhaps most importantly, in our recent development priorities survey our use of Phabricator was the most common complaint by a fair margin, both in case of respondents who have contributed patches and those who have not. On the whole, past contributors and potential future contributors alike have strongly indicated that they want a more Git-like experience. Of course, this wasn't terribly surprising; this is just the most recent case where contributors have made this preference known. Frankly, in a sense it is hard to blame them. The fact that users need to install a PHP tool, Arcanist, to contribute anything but documentation patches has always seemed like unnecessary friction to me and I would be quite happy to be rid of it. Indeed we have had a quite healthy number of GitHub documentation patches since we started accepting them. This makes me thing that there may indeed be potential contributoes that we are leaving on the table. # What to do With Rackspace support ending at the end of year, now may be a good time to consider whether we really want to continue on this road. Phabricator is great at code review but I am less and less certain that it is worth the maintenance uncertainty and potential lost contributors that it costs. Moreover, good alternatives seem closer at-hand than they were when we deployed Phabricator. ## Move to GitHub When people complain about our infrastructure, they often use GitHub as the example of what they would like to see. However, realistically I have a hard time seeing GitHub as a viable option. Its feature set is simply insufficient enough to handle the needs of a larger project like GHC without significant external tooling (as seen in the case of Rust-lang). The concrete reasons have been well-documented in previous discussions but, to summarize, * its review functionality is extremely painful to use with larger patches * rebased patches lead to extreme confusion and lost review comments * it lacks support for pre-receive hooks, which serve as a last line of defense to prevent unintended submodule changes * its inability to deal with external tickets is problematic * there is essentially no possibility that we could eventually migrate GHC's tickets to GitHub's ticket tracker without considerable data loss (much less manage the ticket load that would result), meaning that we would forever be stuck with maintaining Trac. * on a personal note, its search functionality has often left me empty-handed On the whole, these issues seem pretty hard to surmount. ## Move to GitLab In using GitLab for another project over the past months I have been positively surprised by its quality. It handles rebased merge requests far better than GitHub, has functional search, and quite a usable review interface. Furthermore, upstream has been extremely responsive to suggestions for improvement [1]. Even out-of-the-box it seems to be flexible enough to accommodate our needs, supporting integration with external issue trackers, having reasonable release management features, and support for code owners to automate review triaging (replacing much of the functionality of Phabricator's Herald). Finally, other FOSS projects' [3] recent migrations from Phabrictor to GitLab have shown that GitLab-the-company is quite willing to offer help when needed. I took some time this weekend to try setting up a quick GHC instance [2] to play around with. Even after just a few hours of playing around I think the result is surprisingly usable. Out of curiosity I also played around with importing some tickets from Trac (building on Matt Pickering's Trac-to-Maniphest migration tool). With relatively little effort I was even able to get nearly all of our tickets (as of 9 months ago) imported while preserving ticket numbers (although there are naturally a few wrinkles that would need to be worked out). Naturally, I think we should treat the question of ticket tracker migration as an orthogonal one to code review, but it is good to know that this is possible. ## Continue with Phabricator Continuing with Phabricator is of course an option. Its review functionality is great and it has served us reasonably well. However, compared to GitLab and even GitHub of today its features seem less distinguished than they once did. Moreover, the prospect of having to maintain a largely stagnant product with no support strikes me as a slightly dangerous game to play. Working around the issues we have recently encountered has already cost a non-negligible amount of time. # The bottom line If it wasn't clear already, I think that we should strongly consider a move to GitLab. At this point it seems clear that it isn't going to vanish, has a steady pace of development, is featureful, and available. However, these are just my thoughts. What do you think? Cheers, - Ben [1] 11.4 will ship with a file tree view in the code review interface, which I reported (https://gitlab.com/gitlab-org/gitlab-ce/issues/46474) as being is one of the Phabricator features I missed the most during review [2] https://gitlab.home.smart-cactus.org/ghc/ghc/issues/14641 [3] The GNOME and freedesktop.org projects have recently migrated, the former from a hodge-podge of self-hosted services and the latter from Phabricator

17 40

GHC's module hierarchy
by Simon Peyton Jones 02 Oct '19

02 Oct '19

Dear ghc-devs hsyl20 proposes a radical overhaul of the module structure of GHC itself. He or she suggested it six months ago in https://ghc.haskell.org/trac/ghc/ticket/13009 and has now offered a monster patch https://phabricator.haskell.org/D3647 It's clearly the result of a lot of work, but I was the only one who responded on the original ticket, and it'll affect all of your lives in a very immediate way. So, would you like to * consider the idea * look at the actual re-mapping of modules hsyl20 proposals * express an opinion about whether to go ahead Probably the ticket, rather than Phab, is the best place to comment on the general idea. I'd like to thank hsyl20. GHC's rather flat module structure has grown incrementally over years. But still, there are pros and cons. Simon

2 1

Any ways to test a GHC build against large set of packages (including test suites)?
by Ömer Sinan Ağacan 25 Jul '19

25 Jul '19

Hi, I'd like to test some GHC builds + some compile and runtime flag combinations against a large set of packages by building them and running test suites. For this I need - A set of packages that are known to work with latest GHC - A way to build them and run their test suites (if I could specify compile and runtime flags that'd be even better) I think stackage can serve as (1) but I don't know how to do (2). Can anyone point me to the right direction? I vaguely remember some nix-based solution for this that was being discussed on the IRC channel, but can't recall any details. Thanks, Ömer

9 15

Using same stdout/stderr files for multiple tests?
by Ömer Sinan Ağacan 27 Jan '19

27 Jan '19

I have a test that I want to run with different compile and runtime parameters. I managed to reuse the source file across different tests by adding a extra_files(['source.hs']) to the tests, but I don't know how to do the same for stdout/stderr files. Any ideas? In more details, I have test.hs test.stdout and two tests test('test', [extra_run_opts('...')], compile_and_run, []) test('test_debug', [extra_run_opts('...'), extra_hc_opts('-debug'), extra_files(['test.hs'])], compile_and_run, []) The first test works fine, but the second test fails because I don't know how to tell it to use test.stdout as the stdout file and it looks for test_debug.stdout. Thanks Ömer

3 4

Default options for -threaded
by lonetiger＠gmail.com 04 Jan '19

04 Jan '19

Hi All, A user on https://ghc.haskell.org/trac/ghc/ticket/11054 has asked why -N -qa isn’t the default for -threaded. I’m afraid I don’t know a good reason why it’s not. Can anyone help shed some light on this? If there is no good reason, Would anyone object to making it the default? Thanks, Tamar

7 10

nofib output difference
by Joachim Breitner 07 Dec '18

07 Dec '18

Hi, currently, perf.haskell.org has stopped because recent commits fail to build with: expected stdout not matched by reality --- fft2.stdout7 2018-11-28 12:00:24.126015465 -0500 +++ /tmp/runtest6392.1 2018-11-28 12:59:59.329423367 -0500 @@ -1,3 +1,3 @@ -result1 = 2.59635799135966e-12 -result2 = 2.59635799135966e-12 -result3 = 4.8279900966008427e-8 +result1 = 2.649891777856828e-12 +result2 = 2.649891777856828e-12 +result3 = 4.830900479646516e-8 make[2]: *** [../../mk/target.mk:102: runtests] Error 1 do others observe this as well? This is worrying, because there has been no change to nofib since it last worked. Can we add a check of the functional correctness of the nofib programs to the CI somehow? Cheers, Joachim -- Joachim Breitner mail(a)joachim-breitner.de http://www.joachim-breitner.de/

3 4

Cmm Memory Model (Understanding #15449)
by Travis Whitaker 07 Dec '18

07 Dec '18

Hello GHC Devs, I'm trying to get my head around ticket #15449 ( https://ghc.haskell.org/trac/ghc/ticket/15449) This gist of things is that GHC generates incorrect aarch64 code that causes memory corruption in multithreaded programs run on out-of-order machines. User trommler discovered that similar issues are present on PowerPC, and indeed ARMv7 and PowerPC support the same types of load/store reorderings. The LLVM code emitted by GHC may be incorrect with respect to LLVM's memory model, but this isn't a problem on architectures with minimal reordering like x86. I had initially thought that GHC simply wasn't emitting the appropriate LLVM fences; there's an elephant-gun-approach here ( https://github.com/TravisWhitaker/ghc/commits/ghc843-wip/T15449) that guards each atomic operation with a full barrier. I still believe that GHC is omitting necessary LLVM fences, but this change is insufficient to fix the behavior of the test case (which is simply GHC itself compiling a test package with '-jN', N > 1). It seems there's a long and foggy history of the Cmm memory model. Edward Yang discusses this a bit in his post here ( http://blog.ezyang.com/2014/01/so-you-want-to-add-a-new-concurrency-primiti…) and issues similar to #15449 have plagued GHC in the past, like #12469 ( https://ghc.haskell.org/trac/ghc/ticket/12469) Worryingly, GHC only has MO_WriteBarrier, whereas PowerPC and ARMv7 really need read, write, and full memory barriers. On ARM an instruction memory barrier might be required as well, but I don't know enough about STG/Cmm to say for sure, and it'd likely be LLVM's responsibility to emit that anyway. I'm hoping that someone with more tribal knowledge than I might be able to give me some pointers with regards to the following areas: - Does STG itself have anything like a memory model? My intuition says 'definitely not', but given that STG expressions may contain Cmm operations (via StgCmmPrim), there may be STG-to-STG transformations that need to care about the target machine's memory model. - With respect to Cmm, what reorderings does GHC perform? What are the relevant parts of the compiler to begin studying? - Are the LLVM atomics that GHC emits correct with respect to the LLVM memory model? As it stands now LLVM fences are only emitted for MO_WriteBarrier. Without fences accompanying the atomics, it seems the LLVM compiler could float dependent loads/stores past atomic operations. - Why is MO_WriteBarrier the only provided memory barrier? My hunch is that it's because this is the only sort of barrier required on x86, which only allows loads to be reordered with older stores, but perhaps I'm missing something? Is it plausible that Cmm simply needs additional barrier primitives to target these weaker memory models? Conversely, is there some property of Cmm that let's us get away without read barriers at all? Naturally, if I've got any of this wrong or are otherwise barking up the wrong tree, please let me know. Thanks for all your efforts! Travis Whitaker

4 3

RE: Windows test failures
by Simon Peyton Jones 06 Dec '18

06 Dec '18

At the end of the first test run it would have given a list of tests that failed and a line saying TEST=".... List of tests..." Copy that line and at the root of the checkout do make TEST=".... List of tests..." test -C testsuite/tests (that's uppercase C). This will run everything using one thread. :) OK, done. Results below. Simon /c/code/HEAD$ make TEST="T10420 T10672_x64 T13385 T14452 T15815 T3307 T3319 T4006 TH_scopedTvs environment001 plugin-recomp-change plugin-recomp-flags plugin-recomp-impure plugin-recomp-pure plugins07 plugins09 plugins10 plugins11 plugins13 plugins14 print017" test -C testsuite/tests make: Entering directory '/c/code/HEAD/testsuite/tests' PYTHON="python3" "python3" ../driver/runtests.py -e "ghc_compiler_always_flags='-dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output'" -e config.compiler_debugged=False -e ghc_with_native_codegen=True -e config.have_vanilla=True -e config.have_dynamic=False -e config.have_profiling=False -e ghc_with_threaded_rts=True -e ghc_with_dynamic_rts=False -e config.have_interp=True -e config.unregisterised=False -e config.have_gdb=False -e config.have_readelf=True -e config.ghc_dynamic_by_default=False -e config.ghc_dynamic=False -e ghc_with_smp=True -e ghc_with_llvm=False -e windows=True -e darwin=False -e config.in_tree_compiler=True -e config.cleanup=True -e config.local=True --rootdir=. --config-file=../config/ghc -e 'config.platform="x86_64-unknown-mingw32"' -e 'config.os="mingw32"' -e 'config.arch="x86_64"' -e 'config.wordsize="64"' -e 'config.timeout=int() or config.timeout' -e 'config.exeext=".exe"' -e 'config.top="/c/code/HEAD/testsuite"' --config 'compiler="/c/code/HEAD/inplace/bin/ghc-stage2.exe"' --config 'ghc_pkg="/c/code/HEAD/inplace/bin/ghc-pkg.exe"' --config 'haddock=' --config 'hp2ps="/c/code/HEAD/inplace/bin/hp2ps.exe"' --config 'hpc="/c/code/HEAD/inplace/bin/hpc.exe"' --config 'gs="gs"' --config 'timeout_prog="../timeout/install-inplace/bin/timeout.exe"' -e "config.stage=2" --rootdir=../../libraries/Win32/tests --rootdir=../../libraries/array/tests --rootdir=../../libraries/base/tests --rootdir=../../libraries/binary/tests --rootdir=../../libraries/bytestring/tests --rootdir=../../libraries/containers/tests --rootdir=../../libraries/deepseq/tests --rootdir=../../libraries/directory/tests --rootdir=../../libraries/filepath/tests --rootdir=../../libraries/ghc-compact/tests --rootdir=../../libraries/ghc-heap/tests --rootdir=../../libraries/ghc-prim/tests --rootdir=../../libraries/haskeline/tests --rootdir=../../libraries/hpc/tests --rootdir=../../libraries/pretty/tests --rootdir=../../libraries/process/tests --rootdir=../../libraries/stm/tests --rootdir=../../libraries/template-haskell/tests --rootdir=../../libraries/text/tests \ --only=T10420 --only=T10672_x64 --only=T13385 --only=T14452 --only=T15815 --only=T3307 --only=T3319 --only=T4006 --only=TH_scopedTvs --only=environment001 --only=plugin-recomp-change --only=plugin-recomp-flags --only=plugin-recomp-impure --only=plugin-recomp-pure --only=plugins07 --only=plugins09 --only=plugins10 --only=plugins11 --only=plugins13 --only=plugins14 --only=print017 \ \ \ \ \ \ Timeout is 300 0 Found 398 .T files... Beginning test run at Fri Nov 30 21:07:17 2018 GMTST ====> Scanning ./ado/all.T ====> Scanning ./annotations/should_compile/all.T ====> Scanning ./annotations/should_compile/T13818/all.T …lots more “Scanning” lines… ====> Scanning ../../libraries/ghc-heap/tests/all.T ====> Scanning ../../libraries/hpc/tests/fork/test.T--- driver/T14452.run/T14452.stdout.normalised 2018-11-30 21:07:36.565566000 +0000 +++ driver/T14452.run/T14452.run.stdout.normalised 2018-11-30 21:07:36.567569500 +0000 @@ -1 +1 @@ --O3 +"-O3" --- libraries/base/tests/T4006.run/T4006.stdout.normalised 2018-11-30 21:15:19.867313900 +0000 +++ libraries/base/tests/T4006.run/T4006.run.stdout.normalised 2018-11-30 21:15:19.870787900 +0000 @@ -1,2 +1,2 @@ It works here - + --- libraries/base/tests/IO/environment001.run/environment001.stdout.normalised 2018-11-30 21:16:00.645817500 +0000 +++ libraries/base/tests/IO/environment001.run/environment001.run.stdout.normalised 2018-11-30 21:16:00.654738900 +0000 @@ -1,7 +1,8 @@ - -Test 1: 3 - -Test 2: 1 + + +Test 1: 9 + +Test 2: 3 ! Test 3: 3 ["a","b"] ====> Scanning ../../libraries/hpc/tests/function/test.T ====> Scanning ../../libraries/hpc/tests/function2/test.T ====> Scanning ../../libraries/hpc/tests/ghc_ghci/test.T ====> Scanning ../../libraries/hpc/tests/raytrace/test.T ====> Scanning ../../libraries/hpc/tests/raytrace/tixs/test.T ====> Scanning ../../libraries/hpc/tests/simple/test.T ====> Scanning ../../libraries/hpc/tests/simple/tixs/test.T ====> Scanning ../../libraries/process/tests/all.T ====> Scanning ../../libraries/process/tests/T9775/all.T ====> Scanning ../../libraries/stm/tests/all.T ====> Scanning ../../libraries/template-haskell/tests/all.T =====> T14452(normal) 1 of 21 [0, 0, 0] cd "driver/T14452.run" && $MAKE -s --no-print-directory T14452 Actual stdout output differs from expected: diff -uw "driver/T14452.run/T14452.stdout.normalised" "driver/T14452.run/T14452.run.stdout.normalised" *** unexpected failure for T14452(normal) =====> T13385(ghci) 2 of 21 [0, 1, 0] cd "ghci/scripts/T13385.run" && HC="/c/code/HEAD/inplace/bin/ghc-stage2.exe" HC_OPTS="-dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output -XRebindableSyntax" "/c/code/HEAD/inplace/bin/ghc-stage2.exe" --interactive -v0 -ignore-dot-ghci -fno-ghci-history +RTS -I0.1 -RTS -fghci-leak-check -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output -XRebindableSyntax < T13385.script =====> print017(ghci) 3 of 21 [0, 1, 0] cd "ghci.debugger/scripts/print017.run" && HC="/c/code/HEAD/inplace/bin/ghc-stage2.exe" HC_OPTS="-dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output " "/c/code/HEAD/inplace/bin/ghc-stage2.exe" --interactive -v0 -ignore-dot-ghci -fno-ghci-history +RTS -I0.1 -RTS -fghci-leak-check -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output -ignore-dot-ghci< print017.script =====> print017(ghci-ext) 3 of 21 [0, 1, 0] cd "ghci.debugger/scripts/print017.run" && HC="/c/code/HEAD/inplace/bin/ghc-stage2.exe" HC_OPTS="-dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output " "/c/code/HEAD/inplace/bin/ghc-stage2.exe" --interactive -v0 -ignore-dot-ghci -fno-ghci-history -fexternal-interpreter +RTS -I0.1 -RTS -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output -ignore-dot-ghci< print017.script =====> plugin-recomp-change(normal) 4 of 21 [0, 1, 0] cd "plugins/plugin-recomp-change.run" && $MAKE -s --no-print-directory -C plugin-recomp package.plugins01 TOP=/c/code/HEAD/testsuite cd "plugins/plugin-recomp-change.run" && $MAKE -s --no-print-directory plugin-recomp-change =====> T10672_x64(normal) 5 of 21 [0, 1, 0] cd "rts/T10672/T10672_x64.run" && $MAKE -s --no-print-directory T10672_x64 Timeout happened...killed process "cd "rts/T10672/T10672_x64.run" && $MAKE -s --no-print-directory T10672_x64 "... Wrong exit code for T10672_x64()(expected 0 , actual 99 ) *** unexpected failure for T10672_x64(normal) =====> TH_scopedTvs(normal) 6 of 21 [0, 2, 0] cd "th/TH_scopedTvs.run" && "/c/code/HEAD/inplace/bin/ghc-stage2.exe" -c TH_scopedTvs.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output -XTemplateHaskell -package template-haskell -v0 =====> TH_scopedTvs(ext-interp) 6 of 21 [0, 2, 0] cd "th/TH_scopedTvs.run" && "/c/code/HEAD/inplace/bin/ghc-stage2.exe" -c TH_scopedTvs.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output -XTemplateHaskell -package template-haskell -fexternal-interpreter -v0 =====> T3319(normal) 7 of 21 [0, 2, 0] cd "th/T3319.run" && "/c/code/HEAD/inplace/bin/ghc-stage2.exe" -c T3319.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output -XTemplateHaskell -package template-haskell -ddump-splices -v0 =====> T3319(ext-interp) 7 of 21 [0, 2, 0] cd "th/T3319.run" && "/c/code/HEAD/inplace/bin/ghc-stage2.exe" -c T3319.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output -XTemplateHaskell -package template-haskell -fexternal-interpreter -ddump-splices -v0 =====> T15815(normal) 8 of 21 [0, 2, 0] cd "th/T15815.run" && "/c/code/HEAD/inplace/bin/ghc-stage2.exe" --make T15815B -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output -XTemplateHaskell -package template-haskell -v0 -static =====> T15815(ext-interp) 8 of 21 [0, 2, 0] cd "th/T15815.run" && "/c/code/HEAD/inplace/bin/ghc-stage2.exe" --make T15815B -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output -XTemplateHaskell -package template-haskell -fexternal-interpreter -v0 -static =====> T4006(normal) 9 of 21 [0, 2, 0] cd "libraries/base/tests/T4006.run" && "/c/code/HEAD/inplace/bin/ghc-stage2.exe" -o T4006 T4006.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output cd "libraries/base/tests/T4006.run" && ./T4006 Actual stdout output differs from expected: diff -uw "libraries/base/tests/T4006.run/T4006.stdout.normalised" "libraries/base/tests/T4006.run/T4006.run.stdout.normalised" *** unexpected failure for T4006(normal) =====> T3307(normal) 10 of 21 [0, 3, 0] cd "libraries/base/tests/IO/T3307.run" && $MAKE -s --no-print-directory T3307-test Wrong exit code for T3307()(expected 0 , actual 2 ) Stdout ( T3307 ): Test 1 [99,104,105,110,101,115,101,45,102,105,108,101,45,229,176,143,232,175,180] Ni hao Test 2 [99,104,105,110,101,115,101,45,102,105,108,101,45,229,176,143,232,175,180] Ni hao Test 3 [99,104,105,110,101,115,101,45,102,105,108,101,45,23567,35828] Stderr ( T3307 ): *** framework failure for T3307(normal) 'latin-1' codec can't encode characters in position 24-25: ordinal not in range(256) =====> environment001(normal) 11 of 21 [0, 3, 1] cd "libraries/base/tests/IO/environment001.run" && $MAKE -s --no-print-directory environment001-test Actual stdout output differs from expected: diff -uw "libraries/base/tests/IO/environment001.run/environment001.stdout.normalised" "libraries/base/tests/IO/environment001.run/environment001.run.stdout.normalised" *** unexpected failure for environment001(normal) =====> plugins07(normal) 12 of 21 [0, 4, 1] cd "plugins/plugins07.run" && $MAKE -s --no-print-directory -C rule-defining-plugin package.plugins07 TOP=/c/code/HEAD/testsuite cd "plugins/plugins07.run" && $MAKE -s --no-print-directory plugins07 =====> plugins09(normal) 13 of 21 [0, 4, 1] cd "plugins/plugins09.run" && $MAKE -s --no-print-directory -C simple-plugin package.plugins09 TOP=/c/code/HEAD/testsuite cd "plugins/plugins09.run" && $MAKE -s --no-print-directory plugins09 Actual stdout output differs from expected: diff -uw "plugins/plugins09.run/plugins09.stdout.normalised" "plugins/plugins09.run/plugins09.run.stdout.normalised"--- plugins/plugins09.run/plugins09.stdout.normalised 2018-11-30 21:17:20.709370700 +0000 +++ plugins/plugins09.run/plugins09.run.stdout.normalised 2018-11-30 21:17:20.711314000 +0000 @@ -1,9 +0,0 @@ -parsePlugin(a,b) -interfacePlugin: Prelude -interfacePlugin: GHC.Float -interfacePlugin: GHC.Base -typeCheckPlugin (rn) -interfacePlugin: GHC.Types -typeCheckPlugin (tc) -interfacePlugin: GHC.Integer.Type -interfacePlugin: GHC.Natural --- plugins/plugins10.run/plugins10.stdout.normalised 2018-11-30 21:17:57.644357800 +0000 +++ plugins/plugins10.run/plugins10.run.stdout.normalised 2018-11-30 21:17:57.646078700 +0000 @@ -1,21 +0,0 @@ -parsePlugin() -interfacePlugin: Prelude -interfacePlugin: Language.Haskell.TH -interfacePlugin: Language.Haskell.TH.Quote -interfacePlugin: GHC.Float -interfacePlugin: GHC.Base -interfacePlugin: Language.Haskell.TH.Syntax -typeCheckPlugin (rn) -interfacePlugin: GHC.Types -typeCheckPlugin (tc) -interfacePlugin: GHC.Integer.Type -interfacePlugin: GHC.Natural -parsePlugin(a) -typeCheckPlugin (rn) -interfacePlugin: Language.Haskell.TH.Lib.Internal -metaPlugin: return [] -metaPlugin: quoteExp stringify "x" -interfacePlugin: GHC.CString -typeCheckPlugin (rn) -typeCheckPlugin (rn) -typeCheckPlugin (tc) *** unexpected failure for plugins09(normal) =====> plugins10(normal) 14 of 21 [0, 5, 1] cd "plugins/plugins10.run" && $MAKE -s --no-print-directory -C simple-plugin package.plugins10 TOP=/c/code/HEAD/testsuite cd "plugins/plugins10.run" && $MAKE -s --no-print-directory plugins10 Actual stdout output differs from expected: diff -uw "plugins/plugins10.run/plugins10.stdout.normalised" "plugins/plugins10.run/plugins10.run.stdout.normalised" *** unexpected failure for plugins10(normal) =====> plugins11(normal) 15 of 21 [0, 6, 1] cd "plugins/plugins11.run" && $MAKE -s --no-print-directory -C simple-plugin package.plugins11 TOP=/c/code/HEAD/testsuite cd "plugins/plugins11.run" && $MAKE -s --no-print-directory plugins11 Timeout happened...killed process "cd "plugins/plugins11.run" && $MAKE -s --no-print-directory plugins11 "... Wrong exit code for plugins11()(expected 0 , actual 99 ) *** unexpected failure for plugins11(normal) =====> plugins13(normal) 16 of 21 [0, 7, 1] cd "plugins/plugins13.run" && $MAKE -s --no-print-directory -C simple-plugin package.plugins13 TOP=/c/code/HEAD/testsuite cd "plugins/plugins13.run" && $MAKE -s --no-print-directory plugins13 =====> plugins14(normal) 17 of 21 [0, 7, 1] cd "plugins/plugins14.run" && $MAKE -s --no-print-directory -C simple-plugin package.plugins14 TOP=/c/code/HEAD/testsuite cd "plugins/plugins14.run" && $MAKE -s --no-print-directory plugins14 =====> T10420(normal) 18 of 21 [0, 7, 1] cd "plugins/T10420.run" && $MAKE -s --no-print-directory -C rule-defining-plugin package.T10420 TOP=/c/code/HEAD/testsuite cd "plugins/T10420.run" && $MAKE -s --no-print-directory T10420 =====> plugin-recomp-pure(normal) 19 of 21 [0, 7, 1] cd "plugins/plugin-recomp-pure.run" && $MAKE -s --no-print-directory -C plugin-recomp package.plugins01 TOP=/c/code/HEAD/testsuite cd "plugins/plugin-recomp-pure.run" && $MAKE -s --no-print-directory plugin-recomp-pure Timeout happened...killed process "cd "plugins/plugin-recomp-pure.run" && $MAKE -s --no-print-directory plugin-recomp-pure "... Wrong exit code for plugin-recomp-pure()(expected 0 , actual 99 ) Stdout ( plugin-recomp-pure ): [1 of 1] Compiling Main ( plugin-recomp-test.hs, plugin-recomp-test.o ) Stderr ( plugin-recomp-pure ): Simple Plugin Passes Queried Got options: Simple Plugin Pass Run *** unexpected failure for plugin-recomp-pure(normal) =====> plugin-recomp-impure(normal) 20 of 21 [0, 8, 1] cd "plugins/plugin-recomp-impure.run" && $MAKE -s --no-print-directory -C plugin-recomp package.plugins01 TOP=/c/code/HEAD/testsuite cd "plugins/plugin-recomp-impure.run" && $MAKE -s --no-print-directory plugin-recomp-impure Wrong exit code for plugin-recomp-impure()(expected 0 , actual 2 ) Stdout ( plugin-recomp-impure ): [1 of 1] Compiling Main ( plugin-recomp-test.hs, plugin-recomp-test.o ) Linking plugin-recomp-test.exe ... [1 of 1] Compiling Main ( plugin-recomp-test.hs, plugin-recomp-test.o ) [Plugin forced recompilation] Stderr ( plugin-recomp-impure ): Simple Plugin Passes Queried Got options: Simple Plugin Pass Run Simple Plugin Passes Queried Access violation in generated code when reading 0xa824 Attempting to reconstruct a stack trace... Frame Code address * 0x49adab0 0x1eb49ec C:\code\HEAD\inplace\bin\ghc-stage2.exe+0x1ab49ec make[1]: *** [Makefile:99: plugin-recomp-impure] Error 11 *** unexpected failure for plugin-recomp-impure(normal) =====> plugin-recomp-flags(normal) 21 of 21 [0, 9, 1] cd "plugins/plugin-recomp-flags.run" && $MAKE -s --no-print-directory -C plugin-recomp package.plugins01 TOP=/c/code/HEAD/testsuite cd "plugins/plugin-recomp-flags.run" && $MAKE -s --no-print-directory plugin-recomp-flags Traceback (most recent call last): File "/c/code/HEAD/testsuite/driver/testlib.py", line 824, in test_common_work do_test(name, way, func, args, files) File "/c/code/HEAD/testsuite/driver/testlib.py", line 910, in do_test result = func(*[name,way] + args) File "/c/code/HEAD/testsuite/driver/testlib.py", line 993, in run_command return simple_run( name, '', override_options(cmd), '' ) File "/c/code/HEAD/testsuite/driver/testlib.py", line 1338, in simple_run dump_stderr(name) File "/c/code/HEAD/testsuite/driver/testlib.py", line 1501, in dump_stderr print(str) UnicodeEncodeError: 'latin-1' codec can't encode characters in position 24-25: ordinal not in range(256) Unexpected results from: TEST="T10672_x64 T14452 T3307 T4006 environment001 plugin-recomp-impure plugin-recomp-pure plugins09 plugins10 plugins11" SUMMARY for test run started at Fri Nov 30 21:07:17 2018 GMTST 0:26:45 spent to go through 21 total tests, which gave rise to 36 test cases, of which 11 were skipped 0 had missing libraries 15 expected passes 0 expected failures 1 caused framework failures 0 caused framework warnings 0 unexpected passes 9 unexpected failures 0 unexpected stat failures Unexpected failures: driver/T14452.run T14452 [bad stdout] (normal) rts/T10672/T10672_x64.run T10672_x64 [bad exit code] (normal) libraries/base/tests/T4006.run T4006 [bad stdout] (normal) libraries/base/tests/IO/environment001.run environment001 [bad stdout] (normal) plugins/plugins09.run plugins09 [bad stdout] (normal) plugins/plugins10.run plugins10 [bad stdout] (normal) plugins/plugins11.run plugins11 [bad exit code] (normal) plugins/plugin-recomp-pure.run plugin-recomp-pure [bad exit code] (normal) plugins/plugin-recomp-impure.run plugin-recomp-impure [bad exit code] (normal) Framework failures: libraries/base/tests/IO/T3307.run T3307 [normal] ('latin-1' codec can't encode characters in position 24-25: ordinal not in range(256)) Appending 0 stats to git notes. make: *** [../mk/test.mk:342: test] Error 1 make: Leaving directory '/c/code/HEAD/testsuite/tests' /c/code/HEAD$ From: Phyx <lonetiger(a)gmail.com> Sent: 30 November 2018 11:58 To: Simon Peyton Jones <simonpj(a)microsoft.com> Subject: Re: Windows test failures At the end of the first test run it would have given a list of tests that failed and a line saying TEST=".... List of tests..." Copy that line and at the root of the checkout do make TEST=".... List of tests..." test -C testsuite/tests (that's uppercase C). This will run everything using one thread. :) Should give a good starting point as to what's wrong. Kind regards, Tamar On Fri, Nov 30, 2018, 10:30 Simon Peyton Jones <simonpj(a)microsoft.com<mailto:simonpj@microsoft.com>> wrote: So I just say make TEST is that right? Or is it make THREADS=1 or what? Sorry to be dim From: Phyx <lonetiger(a)gmail.com<mailto:lonetiger@gmail.com>> Sent: 30 November 2018 08:05 To: Simon Peyton Jones <simonpj(a)microsoft.com<mailto:simonpj@microsoft.com>> Cc: ghc-devs <ghc-devs(a)haskell.org<mailto:ghc-devs@haskell.org>> Subject: Re: Windows test failures Hi Simon, Could you try rerunning those failing tests with make TEST? No threads? I have been trying to clean up the testsuite and there should be only 3 failing tests, two plugin tests with bad stdout and T10672_x64 which I am looking into. I have however noticed that the testsuite has become increasingly unstable under threads for some reason. I also haven't run trunk yet since last week so they could be new.. But rerunning the fails with no threads should give a better idea. Regards, Tamar On Fri, Nov 30, 2018, 07:46 Simon Peyton Jones via ghc-devs <ghc-devs(a)haskell.org<mailto:ghc-devs@haskell.org>> wrote: Ben, Phyx, and other CI folk ‘sh validate –fast’ still gives a lot of failures on Window. The output is below. I would be so good to get this to zero! (another minor thing is that it would be great to eliminate the long path at the beginning – this is platform independent – I have to delete manually many times each day. Thanks Simon Unexpected passes: /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/rts/T9405.run T9405 [unexpected] (normal) Unexpected failures: /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/driver/T14452.run T14452 [bad stdout] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/ghci/linking/ghcilink001.run ghcilink001 [bad exit code] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/ghci/prog010/ghci.prog010.run ghci.prog010 [bad exit code] (ghci) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/ghci/scripts/ghci050.run ghci050 [bad exit code] (ghci) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/ghci/should_run/T14963a.run T14963a [bad exit code] (ghci) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/ghci.debugger/scripts/print012.run print012 [bad exit code] (ghci) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/ghci.debugger/scripts/print016.run print016 [bad exit code] (ghci) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/ghci.debugger/scripts/break019.run break019 [bad exit code] (ghci) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/ghci.debugger/scripts/break028.run break028 [bad exit code] (ghci) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/ghci.debugger/scripts/dynbrk002.run dynbrk002 [bad exit code] (ghci) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/ghci.debugger/scripts/T2740.run T2740 [bad exit code] (ghci) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/indexed-types/should_fail/T7102a.run T7102a [bad exit code] (ghci) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/plugins/plugin-recomp-change.run plugin-recomp-change [bad exit code] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/rts/T10672/T10672_x64.run T10672_x64 [bad exit code] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/th/TH_spliceGuard.run TH_spliceGuard [exit code non-0] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/th/TH_overlaps.run TH_overlaps [exit code non-0] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/th/T5886.run T5886 [exit code non-0] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/th/T10697_decided_3.run T10697_decided_3 [exit code non-0] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/th/T11629.run T11629 [exit code non-0] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/libraries/base/tests/T4006.run T4006 [bad stdout] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/libraries/base/tests/IO/environment001.run environment001 [bad stdout] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/ghci/should_run/T15633b.run T15633b [bad exit code] (ghci) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/plugins/plugins07.run plugins07 [bad exit code] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/plugins/plugins09.run plugins09 [bad stdout] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/plugins/plugins10.run plugins10 [bad stdout] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/plugins/plugins11.run plugins11 [bad stdout] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/plugins/T10420.run T10420 [bad exit code] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/plugins/plugin-recomp-pure.run plugin-recomp-pure [bad exit code] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/plugins/plugin-recomp-impure.run plugin-recomp-impure [bad exit code] (normal) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/plugins/plugin-recomp-flags.run plugin-recomp-flags [bad exit code] (normal) Framework failures: /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/perf/compiler/MultiLayerModules.run MultiLayerModules [runTest] (Unhandled exception during cleanup: Unable to remove folder '/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/perf/compiler/MultiLayerModules.run': [Errno 13] Permission denied: '/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/perf/compiler/MultiLayerModules.run' Unable to start current test.) /c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test spaces/libraries/base/tests/IO/T3307.run T3307 [normal] ('latin-1' codec can't encode characters in position 24-25: ordinal not in range(256)) _______________________________________________ ghc-devs mailing list ghc-devs(a)haskell.org<mailto:ghc-devs@haskell.org> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskel…>

3 10

Re: [Haskell] Treatment of unknown pragmas
by Artem Pelenitsyn 02 Dec '18

02 Dec '18

Hello Daniel, Annotations API was discussed earlier in this thread. Main points against are: Neil: Significant compilation performance penalty and extra recompilation. ANN pragmas is what HLint currently uses. Brandon: The problem with ANN is it's part of the plugins API, and as such does things like compiling the expression into the program in case a plugin generates code using its value, plus things like recompilation checking end up assuming plugins are in use and doing extra checking. Using it as a compile-time pragma is actually fairly weird from that standpoint. -- Best, Artem On Sat, 27 Oct 2018 at 22:12 Daniel Wagner <dmwit(a)dmwit.com> wrote: > I don't have a really strong opinion, but... isn't this (attaching > string-y data to source constructs) pretty much exactly what GHC's > annotation pragma is for? > ~d > > On Tue, Oct 16, 2018 at 3:14 PM Ben Gamari <ben(a)smart-cactus.org> wrote: > >> Vladislav Zavialov <vlad.z.4096(a)gmail.com> writes: >> >> > What about introducing -fno-warn-pragma=XXX? People who use HLint will >> > add -fno-warn-pragma=HLINT to their build configuration. >> > >> A warning flag is an interesting way to deal with the issue. On the >> other hand, it's not great from an ergonomic perspective; afterall, this >> would mean that all users of HLint (and any other tool requiring special >> pragmas) include this flag in their build configuration. A typical >> Haskell project already needs too much such boilerplate, in my opinion. >> >> I think it makes a lot of sense to have a standard way for third-parties >> to attach string-y information to Haskell source constructs. While it's >> not strictly speaking necessary to standardize the syntax, doing >> so minimizes the chance that tools overlap and hopefully reduces >> the language ecosystem learning curve. >> >> Cheers, >> >> - Ben >> _______________________________________________ >> > Haskell mailing list >> Haskell(a)haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell >> > _______________________________________________ > Haskell mailing list > Haskell(a)haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell >

2 1

typePrimRep invariants
by Csaba Hruska 01 Dec '18

01 Dec '18

Hi, I'd like to export the PrimRep of every binder and data con from Haskell modules. I have a modified GHC 8.6 which serializes the PrimRep for every binder during the compilation. It uses the *typePrimRep* function. When my customised GHC compiles the base library it always thows the following error for the *libraries/base/GHC/Err.hs* <https://github.com/grin-tech/ghc-grin/blob/master/ghc-8.6.2/libraries/base/…> module. 1. Does that mean that there is hidden requirement for typePrimRep input? 2. If so what are the restrictions? 3. Can you see anything in the Err module source code that could not be a proper input for typePrimRep function? Thanks, Csaba "inplace/bin/ghc-stage1" -hisuf hi -osuf o -hcsuf hc -static -H32m -O -Wall -this-unit-id base-4.12.0.0 -hide-all-packages -i -ilibraries/base/. -ilibraries/base/dist-install/build -Ilibraries/base/dist-install/build -ilibraries/base/dist-install/build/./autogen -Ilibraries/base/dist-install/build/./autogen -Ilibraries/base/include -Ilibraries/base/dist-install/build/include -optP-DOPTIMISE_INTEGER_GCD_LCM -optP-include -optPlibraries/base/dist-install/build/./autogen/cabal_macros.h -package-id ghc-prim-0.5.3 -package-id integer-gmp-1.0.2.0 -package-id rts -this-unit-id base -Wcompat -Wnoncanonical-monad-instances -XHaskell2010 -O2 -haddock -no-user-package-db -rtsopts -Wno-trustworthy-safe -Wno-deprecated-flags -Wnoncanonical-monad-instances -odir libraries/base/dist-install/build -hidir libraries/base/dist-install/build -stubdir libraries/base/dist-install/build -split-sections -dynamic-too -c libraries/base/./GHC/Err.hs -o libraries/base/dist-install/build/GHC/Err.o -dyno libraries/base/dist-install/build/GHC/Err.dyn_o ghc-stage1: panic! (the 'impossible' happened) (GHC version 8.6.2 for x86_64-unknown-linux): runtimeRepPrimRep typePrimRep (a_a1ML :: TYPE r_a1MK) r_a1MK Call stack: CallStack (from HasCallStack): callStackDoc, called at compiler/utils/Outputable.hs:1160:37 in ghc:Outputable pprPanic, called at compiler/simplStg/RepType.hs:358:5 in ghc:RepType Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug libraries/base/ghc.mk:4: recipe for target 'libraries/base/dist-install/build/GHC/Err.o' failed make[1]: *** [libraries/base/dist-install/build/GHC/Err.o] Error 1 Makefile:122: recipe for target 'all' failed make: *** [all] Error 2

1 2