October 2014 - ghc-devs

Problems compiling with llvm-3.5.0-2 on ARM
by Jarl Gunnar Flaten 07 Nov '14

07 Nov '14

Hello, (cf. reddit thread <http://www.reddit.com/r/haskell/comments/2kpsz5/ghc_problem_new_llvm_schedu…> ) I am trying to compile a simple "hello world" program (test.hs). When compiling I am notified: > [1 of 1] Compiling Main ( test.hs, test.o ) > You are using a new version of LLVM that hasn't been tested yet! > We will try though... > Linking test ... But it seems to finish compiling fine, without further errors. However, running the program (./test): > test: schedule: re-entered unsafely. > Perhaps a 'foreign import unsafe' should be 'safe'? Isn't working. I'm not familiar enough with neither LLVM nor Haskell to troubleshoot from either of these; I don't even know if the messages are even related. Any ideas for solutions? Any commands or tests I can perform to give you more information? Thanks, Jarl Gunnar T. Flaten

4 4

STM and GC write barrier
by Ryan Yates 06 Nov '14

06 Nov '14

Hi All, I'm starting to get to the bottom of something that has been puzzling me for a while. Recently commit 6d784c43592290ec16db8b7f0f2a012dff3ed497 [1] introduced the GC write barrier for TVars. Not fully understanding things and reading the commit message to be saying that this was an optimization, I implemented my hardware transactional memory support without the write barrier (avoiding the extra work inside a transaction). This resulted in occasional crashes where a TVar which was point to a valid heap object when it was committed, pointed to garbage later. My understanding was that the TVar ended up in a later generation then the value that it came to point to and without getting added to the mut list, the value was collected. Adding the write barrier to all TVars written in my hardware transaction made the problem go away. While this all makes sense to me, what doesn't make as much sense is how STM prior to the commit above was not susceptible to the same problem. Is there some machinery to avoid this issue that I'm still missing? Ryan [1]: https://github.com/ghc/ghc/commit/6d784c43592290ec16db8b7f0f2a012dff3ed497

2 2

Tests with compilation errors
by Gintautas Miliauskas 05 Nov '14

05 Nov '14

Going through some validate.sh results, I found compilation errors due to missing libraries, like this one: =====> stm052(normal) 4088 of 4108 [0, 21, 0] cd ../../libraries/stm/tests && 'C:/msys64/home/Gintas/ghc/bindisttest/install dir/bin/ghc.exe' -fforce-recomp -dcore-lint -dcmm-lint -dno-debug-output -no-user-package-db -rtsopt s -fno-warn-tabs -fno-ghci-history -o stm052 stm052.hs -package stm >stm052.comp.stderr 2>&1 Compile failed (status 256) errors were: stm052.hs:10:8: Could not find module ‘System.Random’ Use -v to see a list of the files searched for. I was surprised to see that these are not listed in the test summary at the end of the test run, but only counted towards the "X had missing libraries" row. That setup makes it really easy to miss them, and I can't think of a good reason to sweep such tests under the rug; a broken test is a failing test. How about at least listing such failed tests in the list of failed tests of the end? At least in this case the error does not seem to be due to some missing external dependencies (which probably would not be a great idea anyway...). The test does pass if I remove the "-no-user-package-db" argument. What was the intention here? Does packaging work somehow differently on Linux? (I'm currently testing on Windows.) On a related note, how about separating test failures from failing performance tests ("stat too good" / "stat not good enough")? The latter are important, but they seem to be much more prone to fail without good reason. Perhaps do some color coding of the test runner output? That would also help. -- Gintautas Miliauskas

4 6

GHC on Windows (extended/broad discussion)
by Austin Seipp 03 Nov '14

03 Nov '14

Gintautas, Tamar, Roman, (CC'ing those on https://ghc.haskell.org/trac/ghc/wiki/WindowsTaskForce, and Kyrill, who has helped us out much in the past) Thank you all for all your help with Windows recently. I apologize for not responding to some of your concerns sooner in the recent threads about tarballs, etc. First off, all your contributions are extremely welcome - GHC has had many talented Windows hackers in days long past, but these days this number has dwindled! Anyone who has an interest in GHC on Windows is in a place to make a big impact and help us. All the work Gintautas has done for example, will dramatically improve the ghc-tarballs scenario. On that note: Gintautas, I will get D339 merged in ASAP, as soon as I test it and make a download mirror for you. Haskell.org has an awesome new CDN setup, and once I implement https://downloads.haskell.org, it will be easy to update tarballs and serve them to mass amounts of users. However, beyond that, we still need more done. First off, if you can help, we can help you! We can make lots of Windows build bots for people on demand, so if you're in desperate need of disk space or your computers are a bit slow, we can help accommodate. Right now, we have nightly builds with Gabor's[1] build system, and soon, we're working on a Phabricator integration, which should be great - and hopefully reduce the amount of breakage substantially. I also notice there is a ticket list of Windows issues[2], and that's fantastic. After a quick glance, a lot of these tickets are old, duplicates, or could possibly be closed or fixed easily. A good first task for any new contributor would be to go through this list, and try to replicate some of them! And you can always ask me - I can certainly help you navigate GHC a bit to get somewhere. But there are still other things. The Win32 package for example, is dreadfully lacking in maintainership. While we merge patches, it would be great to see a Windows developer spearhead and clean it up - we could even make some improvements in GHC itself based on this. This would be an excellent opportunity to make a good impact in the broader ecosystem! Finally, we desperately need someone to consult with when we're up a creek. Are certain patches OK for Windows? What's the best way to fix certain bugs, or implement certain features? I feel like often we try to think about this, but it's a bit lonely when nobody else is there to help! I'm not sure how to fix this, other than encouraging things like doing active code reviews and helping grind out some patches. But at the very minimum, I'd just like to talk with you about things perhaps! So in summary - the work so far is grand, and we want to help you do more! And I'm sure everyone can help - there's always so much to do and so little time, we need to encourage it all we can. As Simon says: Upward and Onward! [1] http://haskell.inf.elte.hu/builders/ [2] https://ghc.haskell.org/trac/ghc/query?status=!closed&os=Windows&desc=1&ord… -- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

8 18

package hashes
by Simon Peyton Jones 03 Nov '14

03 Nov '14

Edward On branch wip/new-flatten-skolems-Oct14, I'm getting this test failure repeatably on test safePkgO1. I'm pretty sure I have done nothing to mess with package hashes! Any ideas? Simon =====> safePkg01(normal) 108 of 120 [0, 0, 0] cd ./check/pkg01 && $MAKE -s --no-print-directory safePkg01 VANILLA=--enable-library-vanilla PROF=--disable-library-profiling DYN=--enable-shared </dev/null >safePkg01.run.stdout 2>safePkg01.run.stderr Actual stdout output differs from expected: --- ./check/pkg01/safePkg01.stdout 2014-10-29 15:09:16.000000000 +0000 +++ ./check/pkg01/safePkg01.run.stdout 2014-10-30 15:25:17.799094762 +0000 @@ -29,17 +29,17 @@ require own pkg trusted: True M_SafePkg6 -package dependencies: array-0.5.0.1@array_<HASH> +package dependencies: array-0.5.0.1 base-4.8.0.0* trusted: trustworthy require own pkg trusted: False M_SafePkg7 -package dependencies: array-0.5.0.1@array_<HASH> +package dependencies: array-0.5.0.1 base-4.8.0.0* trusted: safe require own pkg trusted: False M_SafePkg8 -package dependencies: array-0.5.0.1@array_<HASH> +package dependencies: array-0.5.0.1 base-4.8.0.0 trusted: trustworthy require own pkg trusted: False

2 1

Core libraries bug tracker
by Simon Peyton Jones 02 Nov '14

02 Nov '14

Edward, and core library colleagues, This came up in our weekly GHC discussion * Does the Core Libraries Committee have a Trac? Surely, surely you should, else you'll lose track of issues. * Would you like to use GHC's Trac for the purpose? Advantages: o People often report core library issues on GHC's Trac anyway, so telling them to move it somewhere else just creates busy-work --- and maybe they won't bother, which leaves it in our pile. o Several of these libraries are closely coupled to GHC, and you might want to milestone some library tickets with an upcoming GHC release * If so we'd need a canonical way to identify tickets as CLC issues. Perhaps by making "core-libraries" the owner? Or perhaps the "Component" field? * Some core libraries (e.g. random) have a maintainer that isn't the committee. So that maintainer should be the owner of the ticket. Or the CLC might like a particular member to own a ticket. Either way, that suggest using the "Component" field to identify CLC tickets * Or maybe you want a Trac of your own? The underlying issue from our end is that we'd like a way to * filter out tickets that you are dealing with * and be sure you are dealing with them * without losing track of milestones... i.e. when building a release we want to be sure that important tickets are indeed fixed before releasing Simon

4 8

GHC Weekly News - 2014/10/31 (Halloween Edition)
by Austin Seipp 31 Oct '14

31 Oct '14

Hello *, Welcome to the GHC Weekly news. And it's just in time before you go out and eat lots of candy and food. * David Feuer and Joachim Brietner spent some time this past week talking about more optimizations for Haskell code for fusing code, and creating better consumers and producers. This work includes optimizations of "One shot lambdas" (lambdas used at most once) and Call Arity, which was implemented by Joachim at Microsoft Research. The thread is here - https://www.haskell.org/pipermail/ghc-devs/2014-October/006901.html The current situation is that Call Arity and One shot analysis tends to have good combined results for exploiting more fusion opportunities, but sometimes these backfire. As a result, Joachim and David have worked on improving the situation - particularly by letting programmers help with a new `oneShot` primitive (in Phab:D392 & Phab:D393). * Herbert Valerio Riedel opened up a discussion about the origins of code contributions. In short, we'd like to have some stronger ground to stand on in the face of contributions from contributors - the origin of a change and its legal status is a bit nebulous. The thread is here: https://www.haskell.org/pipermail/ghc-devs/2014-October/006959.html Overall, there's been a lot of separate points made, including CLAs (unlikely), "Developer Certificates of Origin" a la the Linux Kernel, and reworking how we mark up header files, and keep track of GHC's long history of authors. If you work on a big project where some of these concerns are real, we'd like to hear what you have to say! * Gintautas Milauskas has done some fantastic work for GHC on Windows lately, including fixing tests, improving the build, and making things a lot more reasonable to use. With his work, we hope GHC 7.10 will finally ship an updated MinGW compiler (a long requested feature), and have a lot of good fixes for windows. Thank you, Gintautas! * And on that note, the call for Windows developers rages on - it looks like Gintautaus, Austin, Simon, and others will be meeting to discuss the best way to tackle our current woes. Are you a Windows user? Please give us input - having input is a crucial part of the decision making process, so let us know. * Jan Stolarek had a question about core2core - a lot of questions, in fact. What's the difference between demand, strictness, and cardinality analylsis? Does the demand analyzer change things? And what's going on in some of the implementation? A good read if you're interested in deep GHC optimization magic: https://www.haskell.org/pipermail/ghc-devs/2014-October/006968.html * Peter Wortmann has put up the new DWARF generation patches for review, in Phab:D396. This is one of the major components we still plan on landing in 7.10, and with a few weeks to spare, it looks like we can make sure it's merged for the STABLE freeze! * There have been a lot of good changes in the tree this past week: Thanks to Michael Orlitzky, we plan on adding doctest examples to more modules in 7.10, and increase that coverage further. This is *really* important work, but very low fruit - thanks a ton Michael! `Data.Bifunctor` is now inside base! (Phab:D336) `atomicModifyIORef'` has been optimized with excellent speedups (as much as 1.7x to 1.4x, depending on the RTS used), thanks to some older work by Patrick Palka (Phab:D315). GHC's internals have been reworked to unwire `Integer` from GHC, leading not only to a code cleanup, but laying the foundation for further GMP (and non-GMP!) related `Integer` improvements (Phab:D351). David Feuer and Joachim have been relentless in improving fusion opportunities, including the performance of `take`, `isSuffixOf`, and more prelude improvements, spread over nearly half a dozen patches. And this doesn't even include the work on improving `oneShot` or Call Arity! In a slight change to `base` semantics, David Feuer also finally fixed #9236. This is a change that can expose latent bugs in your program (as it did for Haddock), so be sure to test thoroughly with 7.10 (Phab:D327). GHC now has support for a new `__GLASGOW_HASKELL_TH__` macro, primarily useful for testing bootstrap compilers, or compilers which don't support GHCi. And there have been many closed tickets: #9549, #9593, #9720, #9031, #8345, #9439, #9435, #8825, #9006, #9417, #9727, #2104, #9676, #2628, #9510, #9740, #9734, #9367, #9726, #7984, #9230, #9681, #9747, and #9236. -- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

1 0

cabal error: http://hackage.haskell.org/packages/archive/00-index.tar.gz : ErrorMisc "Error HTTP code: 502"
by George Colpitts 31 Oct '14

31 Oct '14

When I type cabal update I get http://hackage.haskell.org/packages/archive/00-index.tar.gz : ErrorMisc "Error HTTP code: 502" Anybody else seeing this? Thanks George

2 2

git question
by Simon Peyton Jones 31 Oct '14

31 Oct '14

Friends I am working on wip/new-flatten-skolems-Oct14. I have pushed lots of patches up to the main repo. Now I want to rebase to clean up. Can I just do a local rebase and then git push? Nothing special about the push? I know that will confuse anyone who is pulling from that branch, but I've warned them! Simon

3 7

Re: Understanding core2core optimisation pipeline
by Jan Stolarek 31 Oct '14

31 Oct '14

I'm resurrecting this 3-month old thread as I have some more questions about cardinality analysis. 1. I'm still a bit confused about terminology. Demand analysis, strictness analysis, cardinality analysis - do these three terms mean exactly the same thing? If no, then what are the differences? 2. First pass of full laziness is followed by floating in. At that stage we have not yet run the demand analysis and yet the code that does the floating-in checks whether a binder is one-shot (FloatIn.okToFloatInside called by FloatIn.fiExpr AnnLam case). This suggests that cardinality analysis is done earlier (but when?) and that demand analysis is not the same thing as cardinality analysis. 3. Does demand analyser perform any transformations? Or does it only annotate Core with demand information that can be used by subsequent passes? 4. BasicTypes module defines: data OneShotInfo = NoOneShotInfo -- ^ No information | ProbOneShot -- ^ The lambda is probably applied at most once | OneShotLam -- ^ The lambda is applied at most once. Do I understand correctly that `NoOneShotInfo` really means no information, ie. a binding annotated with this might in fact be one shot? If so, then do we have means of saying that a binding is certainly not a one-shot binding? 5. What is the purpose of SetLevels.lvlMFE function? Janek > The wiki page just went live: > > https://ghc.haskell.org/trac/ghc/wiki/Commentary/Compiler/Core2CorePipeline > > It's not yet perfect but it should be a good start. > > > Roughtly, a complete run of the simplifier means "run the simplifier > > repeatedly until nothing further happens". The iterations are the > > successive iterations of this loop. Currently there's a (rather arbitrary) > > limit of four such iterations before we give up and declare victory. > > A limit or a default value for that limit? > > To Ilya: > > If you grep for the "late_dmd_anal" option variable in the compiler/simplCore/SimplCore.lhs > > module, you'll see that it triggers a phase close to the endo of getCoreToDo's tasks, which > > contains, in particular, the "CoreDoStrictness" pass. This is the "late" phase. > > The paper said that the late pass is run to detect single-entry thunks and the reason why it is > run late in the pipeline is that if it were run earlier this information could be invalidated > by the transformations. But in the source code I see that this late pass is followed by the > simplifier, which can invalidate the information. Also, the documentation for -flate-dmd-anal > says: "We found some opportunities for discovering strictness that were not visible earlier; > and optimisations like -fspec-constr can create functions with unused arguments which are > eliminated by late demand analysis". This says nothing about single-netry thunks. So, is the > single-entry thunk optimisation performed by GHC? > > Janek

3 4