3 questions regarding profiling in ghc
Hello Sorry for the uninformative subject, but I've got no less than 3 questions about profiling in ghc. Here we go 1) Can I profile my program if I don't have all the libraries it depends on compiled with profiling? ghc says it can't find profiling variants of this and that module in other packages, when I try ghc -prof. I managed to build all the dependencies with profiling, but that being a necessity doesn't look good to me. Maybe there's a way to avoid that? Moreover, as far as I could tell functions from those packages didn't appear in the call graph after +RTS -p profiling. 2) Can I exclude a function from profiling? That probably means not assigning a cost centre to it. Typical case, I think. Database connect function is rather heavyweight (regarding time) compared to the rest of the code, and it takes up to 98% of time. So the rest of the picture is less informative than it could be. 3) Isn't it possible to have -p profiling data of the interrupted (ctrl-c) program? Thanks a lot! -- Daniil Elovkov
Daniil, While you're waiting for an answer from a GHC internals expert, here's my experience as a fellow user.
1) Can I profile my program if I don't have all the libraries it depends on compiled with profiling?
I don't know how to do that, and I don't know how to automatically reinstall all dependencies of my project with profiling enabled. I recently went through reinstalling said dependencies as I found them, iteratively. I could have blown away and reinstalled GHC instead, and saved time. To prevent a recurrence, I now have this in my ~/.cabal/config : library-vanilla: True library-profiling: True This should install both normal and profiling versions of every library that I install with cabal from here out. It's a little slower when installing new library packages, but it doesn't come up often enough to bother me. There may be some pain when I get around to bootstrapping GHC 6.12, if it doesn't install profiling builds of its bundled libs.
Moreover, as far as I could tell functions from those packages didn't appear in the call graph after +RTS -p profiling.
Did you use "-auto-all", to automatically create cost centers for all top-level functions? I find that I get very verbose cost info for definitions under imported libraries.
2) Can I exclude a function from profiling? That probably means not assigning a cost centre to it.
If "-auto-all" doesn't please you, you can manually define your cost centers in your code, leaving out the ones you don't care about. But unless I'm mistaken, that doesn't exclude those costs, but rather includes them in the calling cost center. So it may not be what you're asking for.
Typical case, I think. Database connect function is rather heavyweight (regarding time) compared to the rest of the code, and it takes up to 98% of time. So the rest of the picture is less informative than it could be.
It's your business, but in that case why would you care about the (time) profile of the rest of the code? I wouldn't spend ten seconds time-optimizing anything but that hot spot. If it can't be improved, you're done. To be clear, I'm assuming you're talking about 98% of CPU time, not wall time; I don't think the profiler reports wall time, except maybe in the summary.
3) Isn't it possible to have -p profiling data of the interrupted (ctrl-c) program?
When I ctrl-c out of my program, I get a nice <program>.prof file in the directory where it's running. If you're not getting that, the difference could be OS environment (I'm developing on Linux), or it could be that I'm using happstack and calling a routine that catches the ctrl-c then exits cleanly. It's Happstack.State.waitForTermination; you can probably distill enough from it to get the same effect. http://hackage.haskell.org/packages/archive/happstack-state/0.3.4/doc/html/s... (Pardon the long link.) My main routine spins off threads to do all the work, and the main thread waits on waitForTermination then shuts down. Hope some of this helps. Regards, John Dorsey
| > 1) Can I profile my program if I don't have all the libraries it depends | > on compiled with profiling? I'm afraid not. It's one of the shortcomings of GHC's profiler, but it's hard to overcome. Simon
John Dorsey wrote:
Daniil,
While you're waiting for an answer from a GHC internals expert, here's my experience as a fellow user.
1) Can I profile my program if I don't have all the libraries it depends on compiled with profiling?
I don't know how to do that, and I don't know how to automatically reinstall all dependencies of my project with profiling enabled. I recently went through reinstalling said dependencies as I found them, iteratively. I could have blown away and reinstalled GHC instead, and saved time.
To prevent a recurrence, I now have this in my ~/.cabal/config :
library-vanilla: True library-profiling: True
This should install both normal and profiling versions of every library that I install with cabal from here out. It's a little slower when installing new library packages, but it doesn't come up often enough to bother me. There may be some pain when I get around to bootstrapping GHC 6.12, if it doesn't install profiling builds of its bundled libs.
Moreover, as far as I could tell functions from those packages didn't appear in the call graph after +RTS -p profiling.
Did you use "-auto-all", to automatically create cost centers for all top-level functions? I find that I get very verbose cost info for definitions under imported libraries.
Yeah, I've got it. Modules in packages were done by cabal configure -p. That probably doesn't imply -auto-all.
2) Can I exclude a function from profiling? That probably means not assigning a cost centre to it.
If "-auto-all" doesn't please you, you can manually define your cost centers in your code, leaving out the ones you don't care about. But unless I'm mistaken, that doesn't exclude those costs, but rather includes them in the calling cost center. So it may not be what you're asking for.
Typical case, I think. Database connect function is rather heavyweight (regarding time) compared to the rest of the code, and it takes up to 98% of time. So the rest of the picture is less informative than it could be.
It's your business, but in that case why would you care about the (time) profile of the rest of the code? I wouldn't spend ten seconds time-optimizing anything but that hot spot. If it can't be improved, you're done.
Makes sense. Well, in other circumstances that connect function may either take a lot less time or not run at all. Of course the answer here could be: so profile the program in _that_ case as well. Ok, I was just wondering :)
To be clear, I'm assuming you're talking about 98% of CPU time, not wall time; I don't think the profiler reports wall time, except maybe in the summary.
3) Isn't it possible to have -p profiling data of the interrupted (ctrl-c) program?
When I ctrl-c out of my program, I get a nice <program>.prof file in the directory where it's running. If you're not getting that, the difference could be OS environment (I'm developing on Linux), or it could be that I'm using happstack and calling a routine that catches the ctrl-c then exits cleanly. It's Happstack.State.waitForTermination; you can probably distill enough from it to get the same effect.
http://hackage.haskell.org/packages/archive/happstack-state/0.3.4/doc/html/s...
(Pardon the long link.) My main routine spins off threads to do all the work, and the main thread waits on waitForTermination then shuts down.
Of course, catching ctrl-c and exiting normally alone is definitely sufficient to get the .prof file, because it's just no different than simply exiting. Thanks John, Simon. -- Daniil Elovkov
On Fri, 2009-11-13 at 17:18 +0300, Daniil Elovkov wrote:
Did you use "-auto-all", to automatically create cost centers for all top-level functions? I find that I get very verbose cost info for definitions under imported libraries.
Yeah, I've got it. Modules in packages were done by cabal configure -p. That probably doesn't imply -auto-all.
Right, because you usually do not want to see cost centres for all of the dependent libs (especially not all the way down) to the bottom of the package stack. What we need is somewhat better control in Cabal so you can say, "actually I do want this package to have cost centres". Duncan
participants (4)
-
Daniil Elovkov -
Duncan Coutts -
John Dorsey -
Simon Peyton-Jones