Hi Simon and others, Yes, an update on our holes in GHC project was very due. What I've been working on: - The command line flag -XHoles and as a language pragma {-# LANGUAGE Holes #-} now work - Holes print a warning message with: - The origin of their type variables - Their local environment - Most of the code has been cleaned up to prepare it for submission as a patch. As an example of the message, consider the module: --- {-# LANGUAGE Holes #-} test x = foldr _ x _ --- This prints: --- [1 of 1] Compiling Main ( test2.hs, interpreted ) test2.hs:3:16: Warning: Found hole `_' with type a0 -> b -> b Where: `b' is a rigid type variable bound by the inferred type of test :: b -> b at test2.hs:3:1 `a0' is a free type variable In scope: x :: b test2.hs:3:20: Warning: Found hole `_' with type [a0] Where: `a0' is a free type variable In scope: x :: b Ok, modules loaded: Main --- The 3 currently remaining issues: - Free type variables are not tidied consistently. For every one of these hole warnings, the same TidyEnv is reused, without taking the updates from the other holes into account. I'm pretty sure I know where this happens and how I could fix it. - What I thought would be the local environment doesn't actually seem to be it. The holes store in their origin the result of `getLclTypeEnv' at their location, but as the Note [Bindings with closed types] says, the TopLevelFlag of these don't actually differentiate the top level from the non-top level bindings. I think it would be more helpful to only show the non-top level bindings at the hole's location, any hints about how to obtain just these would be appreciated. - The holes do not have very accurate source location information, like some other errors have. The hole has its origin, ("test2.hs:3:16"), but somehow not something like: "In the expression: folder _ x _, In an equation for `test': test x = foldr _ x _". Help with how that is supposed to work would also be appreciated. I'm attaching a patch of all my changes compared to HEAD as of about a week ago. All my work can also still be found on https://github.com/xnyhps/ghc. Best regards, Thijs Alkemade
During one of my long Agda runs (with GHC-7.4.2), I observed the following output, with run-time options +RTS -S -H11G -M11G -K256M : 7694558208 30623864 3833166176 0.11 0.11 234.75 234.79 0 0 (Gen: 0) 7678904688 29295168 3847737784 0.11 0.11 242.04 242.09 0 0 (Gen: 0) 7662481840 29195736 3861451856 0.11 0.11 249.31 249.35 0 0 (Gen: 0) 7647989280 26482704 3872463688 0.12 0.12 256.64 256.68 0 0 (Gen: 0) 4609865360 25764016 3886000448 0.09 0.09 261.04 261.09 0 0 (Gen: 0) 4581294920 19435032 3891512272 0.07 0.07 265.37 265.42 0 0 (Gen: 0) 4568757088 21095864 3902286000 0.08 0.08 269.70 269.74 0 0 (Gen: 0) 4546421608 21618856 3913923976 0.09 0.09 274.04 274.09 0 0 (Gen: 0) 4521518888 2894668056 3484748224 7.63 7.63 285.94 285.98 0 0 (Gen: 1) 8085358392 23776128 3499185336 0.11 0.11 293.49 293.53 0 0 (Gen: 0) 8064630856 32055112 3515876576 0.13 0.13 300.91 300.95 0 0 (Gen: 0) 8040500112 31477608 3528105088 0.12 0.12 308.37 308.41 0 0 (Gen: 0) 8031456296 29641328 3540632456 0.11 0.11 315.83 315.87 0 0 (Gen: 0) 8018447264 30187208 3554339600 0.12 0.12 323.26 323.31 0 0 (Gen: 0) To my untrained eye, this seems to be saying the following: In the first 4 lines, the heap runs (almost) full before (minor) collections. In lines 5 to 9 it apparently leaves 3G empty before collection, but ``those 3G'' then appear on line 9 in the ``amount of data copied during (major) collection'' column, and after that it runs up to fill all 11G again before the next few minor collections. What is really going on here? (Previously I had never seen such big numbers in the second column on major collections.) Wolfram P.S.: Same effect again, but more dramatic, later during the same Agda run: 448829488 4864536 5710435424 0.02 0.02 1422.80 1422.90 0 0 (Gen: 0) 445544064 3251712 5710248752 0.01 0.01 1423.23 1423.32 0 0 (Gen: 0) 450236784 4148864 5712696848 0.02 0.02 1423.68 1423.77 0 0 (Gen: 0) 445240152 3828120 5713606328 0.02 0.02 1424.10 1424.19 0 0 (Gen: 0) 443285616 5906448 5717731864 0.02 0.02 1424.52 1424.61 0 0 (Gen: 0) 430698248 4773500032 5363214440 9.30 9.30 1434.21 1434.30 0 0 (Gen: 1) 6148455592 13490304 5374609848 0.07 0.07 1439.83 1439.92 0 0 (Gen: 0) 6185350848 27419744 5389326896 0.11 0.11 1445.50 1445.59 0 0 (Gen: 0) 6168805736 23069072 5398725784 0.11 0.11 1451.22 1451.32 0 0 (Gen: 0) 6157744328 23451872 5408370152 0.09 0.09 1456.93 1457.03 0 0 (Gen: 0) 6151715272 25739584 5421044592 0.11 0.11 1462.62 1462.72 0 0 (Gen: 0) 6132589488 24541688 5428809632 0.10 0.10 1468.26 1468.37 0 0 (Gen: 0)
On 17/08/2012 17:08, Wolfram Kahl wrote:
During one of my long Agda runs (with GHC-7.4.2), I observed the following output, with run-time options
+RTS -S -H11G -M11G -K256M
:
7694558208 30623864 3833166176 0.11 0.11 234.75 234.79 0 0 (Gen: 0) 7678904688 29295168 3847737784 0.11 0.11 242.04 242.09 0 0 (Gen: 0) 7662481840 29195736 3861451856 0.11 0.11 249.31 249.35 0 0 (Gen: 0) 7647989280 26482704 3872463688 0.12 0.12 256.64 256.68 0 0 (Gen: 0) 4609865360 25764016 3886000448 0.09 0.09 261.04 261.09 0 0 (Gen: 0) 4581294920 19435032 3891512272 0.07 0.07 265.37 265.42 0 0 (Gen: 0) 4568757088 21095864 3902286000 0.08 0.08 269.70 269.74 0 0 (Gen: 0) 4546421608 21618856 3913923976 0.09 0.09 274.04 274.09 0 0 (Gen: 0) 4521518888 2894668056 3484748224 7.63 7.63 285.94 285.98 0 0 (Gen: 1) 8085358392 23776128 3499185336 0.11 0.11 293.49 293.53 0 0 (Gen: 0) 8064630856 32055112 3515876576 0.13 0.13 300.91 300.95 0 0 (Gen: 0) 8040500112 31477608 3528105088 0.12 0.12 308.37 308.41 0 0 (Gen: 0) 8031456296 29641328 3540632456 0.11 0.11 315.83 315.87 0 0 (Gen: 0) 8018447264 30187208 3554339600 0.12 0.12 323.26 323.31 0 0 (Gen: 0)
To my untrained eye, this seems to be saying the following: In the first 4 lines, the heap runs (almost) full before (minor) collections. In lines 5 to 9 it apparently leaves 3G empty before collection, but ``those 3G'' then appear on line 9 in the ``amount of data copied during (major) collection'' column, and after that it runs up to fill all 11G again before the next few minor collections.
What is really going on here? (Previously I had never seen such big numbers in the second column on major collections.)
It looks like on line 5, the GC thought it was going to do a major collection the next time, so it left 3G free to copy the contents of the old generation. But then it didn't do a major GC until line 9. I've just checked the code, and I think this might be due to a slight inaccuracy in the way that we estimate whether the next GC will be a major one, and at these huge sizes the discrepancy becomes significant. Thanks for pointing it out, I'll fix it to use the same calculation in both places. Cheers, Simon
Wolfram
P.S.: Same effect again, but more dramatic, later during the same Agda run:
448829488 4864536 5710435424 0.02 0.02 1422.80 1422.90 0 0 (Gen: 0) 445544064 3251712 5710248752 0.01 0.01 1423.23 1423.32 0 0 (Gen: 0) 450236784 4148864 5712696848 0.02 0.02 1423.68 1423.77 0 0 (Gen: 0) 445240152 3828120 5713606328 0.02 0.02 1424.10 1424.19 0 0 (Gen: 0) 443285616 5906448 5717731864 0.02 0.02 1424.52 1424.61 0 0 (Gen: 0) 430698248 4773500032 5363214440 9.30 9.30 1434.21 1434.30 0 0 (Gen: 1) 6148455592 13490304 5374609848 0.07 0.07 1439.83 1439.92 0 0 (Gen: 0) 6185350848 27419744 5389326896 0.11 0.11 1445.50 1445.59 0 0 (Gen: 0) 6168805736 23069072 5398725784 0.11 0.11 1451.22 1451.32 0 0 (Gen: 0) 6157744328 23451872 5408370152 0.09 0.09 1456.93 1457.03 0 0 (Gen: 0) 6151715272 25739584 5421044592 0.11 0.11 1462.62 1462.72 0 0 (Gen: 0) 6132589488 24541688 5428809632 0.10 0.10 1468.26 1468.37 0 0 (Gen: 0)
_______________________________________________ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Can you give me read/write access to your github repo? I'm simonpj on github. That way I can add comments/questions in code, and generally clean up. It would make things easier if you could merge with HEAD so that I don't have to mess around moving libraries back in time. ------------------------------ You've put "LANGUAGE Holes" in TcErrors which means I can't bootstrap. ------------------------------ You have this in your patch file, which can't be right + | CHoleCan { + cc_ev :: CtEvidence, + cc_hole_ty :: TcTauType, -- Not a Xi! See same not as above + cc_depth :: SubGoalDepth -- See Note [WorkList] + } + \end{code} \begin{code} @@ -933,6 +940,9 @@ ctPred (CTyEqCan { cc_tyvar = tv, cc_rhs = xi }) ctPred (CFunEqCan { cc_fun = fn, cc_tyargs = xis1, cc_rhs = xi2 }) = mkTcEqPred (mkTyConApp fn xis1) xi2 ctPred (CIrredEvCan { cc_ty = xi }) = xi +ctPred (CHoleCan { cc_flavor = fl, cc_hole_ty = xi }) + = xi + since c_flavor isn't a field of CHoleCan. ----------------------- | The 3 currently remaining issues: | | - Free type variables are not tidied consistently. For every one of | these hole warnings, the same TidyEnv is reused, without taking the | updates from the other holes into account. I'm pretty sure I know where | this happens and how I could fix it. This should be easy. * TcErrors.reportUnsolved uses tyVarsOfWC to find the free type variables of unsolved constraints * It then uses tidyFreeTyVars to assign them names * And that gives an env used in tidying. So it should just work. I hope you are letting the various 'tidy' calls in TcErrors do the work. | - What I thought would be the local environment doesn't actually seem | to be it. The holes store in their origin the result of `getLclTypeEnv' | at their location, but as the Note [Bindings with closed types] says, | the TopLevelFlag of these don't actually differentiate the top level | from the non-top level bindings. I think it would be more helpful to | only show the non-top level bindings at the hole's location, any hints | about how to obtain just these would be appreciated. In this context "local" means "this module" rather than "not top level". Use isExternalName to distinguish top-level things from nested things. | - The holes do not have very accurate source location information, like | some other errors have. The hole has its origin, ("test2.hs:3:16"), but | somehow not something like: "In the expression: folder _ x _, In an | equation for `test': test x = foldr _ x _". Help with how that is | supposed to work would also be appreciated. That's odd. Every Wanted constraint has a WantedLoc (TcRnTypes), which includes a [ErrCtxt], which is that stack of "In ..; In... " stuff you see. Code looks plausible.
Hi Simon,
On Tue, Aug 21, 2012 at 7:14 PM, Simon Peyton-Jones
Can you give me read/write access to your github repo? I'm simonpj on github. That way I can add comments/questions in code, and generally clean up. It would make things easier if you could merge with HEAD so that I don't have to mess around moving libraries back in time.
Done. I've merged with HEAD as of right now. The only subrepository I've forked is the testsuite repository, which you can find on https://github.com/xnyhps/testsuite. I've tried to get validate to pass all tests, I had to fix this one: https://github.com/xnyhps/testsuite/commit/5016899250ace0d2db227569073ad8357..., and I've added this https://github.com/xnyhps/testsuite/commit/47f1bb7f7c8aced6e59f8d3e2413f6c6c... as a test case for holes. It's not a very complicated example of holes, but I'm a bit unsure how to properly write a test case for it. If you have some ideas to do this better, I'd be interested. Some other tests were reporting to be too (in)efficient, but compared to running validate on HEAD without my patch it seemed the same thing happened there, so I haven't look further into those.
------------------------------ You've put "LANGUAGE Holes" in TcErrors which means I can't bootstrap.
Sorry, that was sloppy, I've fixed it.
------------------------------ You have this in your patch file, which can't be right + | CHoleCan { + cc_ev :: CtEvidence, + cc_hole_ty :: TcTauType, -- Not a Xi! See same not as above + cc_depth :: SubGoalDepth -- See Note [WorkList] + } + \end{code}
\begin{code} @@ -933,6 +940,9 @@ ctPred (CTyEqCan { cc_tyvar = tv, cc_rhs = xi }) ctPred (CFunEqCan { cc_fun = fn, cc_tyargs = xis1, cc_rhs = xi2 }) = mkTcEqPred (mkTyConApp fn xis1) xi2 ctPred (CIrredEvCan { cc_ty = xi }) = xi +ctPred (CHoleCan { cc_flavor = fl, cc_hole_ty = xi }) + = xi +
since c_flavor isn't a field of CHoleCan.
This code is gone after merging with HEAD.
-----------------------
| The 3 currently remaining issues: | | - Free type variables are not tidied consistently. For every one of | these hole warnings, the same TidyEnv is reused, without taking the | updates from the other holes into account. I'm pretty sure I know where | this happens and how I could fix it.
This should be easy. * TcErrors.reportUnsolved uses tyVarsOfWC to find the free type variables of unsolved constraints * It then uses tidyFreeTyVars to assign them names * And that gives an env used in tidying.
So it should just work. I hope you are letting the various 'tidy' calls in TcErrors do the work.
I traced the problem back to not zonking the hole constraint properly, reporting them now works as it should (I've removed the hack I was using, and cleaned up some code that didn't need to be monadic anymore). (The reason they weren't zonked properly is that I've given zonkCt a special case for CHoleCans (https://github.com/xnyhps/ghc/blob/eee4242df72cf0a7910e896c3bd6d0fb20b97e37/...), as holes themselves do not carry enough information for me to turn them from CNonCanonicals back into CHoleCans (they become CIrredEvCans). I'm still not sure this is the right thing to do here.)
| - What I thought would be the local environment doesn't actually seem | to be it. The holes store in their origin the result of `getLclTypeEnv' | at their location, but as the Note [Bindings with closed types] says, | the TopLevelFlag of these don't actually differentiate the top level | from the non-top level bindings. I think it would be more helpful to | only show the non-top level bindings at the hole's location, any hints | about how to obtain just these would be appreciated.
In this context "local" means "this module" rather than "not top level". Use isExternalName to distinguish top-level things from nested things.
This works now, thanks!
| - The holes do not have very accurate source location information, like | some other errors have. The hole has its origin, ("test2.hs:3:16"), but | somehow not something like: "In the expression: folder _ x _, In an | equation for `test': test x = foldr _ x _". Help with how that is | supposed to work would also be appreciated.
That's odd. Every Wanted constraint has a WantedLoc (TcRnTypes), which includes a [ErrCtxt], which is that stack of "In ..; In... " stuff you see.
Code looks plausible.
I've fixed the way I was constructing the CtLoc, this was wrong and was always using [] as the [ErrCtxt]. Regards, Thijs Alkemade
participants (4)
-
Simon Marlow -
Simon Peyton-Jones -
Thijs Alkemade -
Wolfram Kahl