#15357: Make nofib suitable for runtime measurements. -------------------------------------+------------------------------------- Reporter: AndreasK | Owner: (none) Type: task | Status: new Priority: normal | Milestone: 8.6.1 Component: NoFib benchmark | Version: 8.4.3 suite | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by AndreasK):
Are you offering to help?! It would be great if so.
there's already FAST/NORM/SLOW setting that is supposed to choose per- benchmarks configurations that take 1s/5s/20s or something vaguely like
I'm currently working on the NCG for Summer of Code so not before that is over. I might get around to it after, or I might never so if someone else picks this up I would be more than grateful. that. Makefiles can set different parameters for the different settings. While helpful to main issue is that: * There are a benchmarks which use the same setting for all three variants. * It doesn't help with keeping runtimes of different benchmarks in the same ballpark. Ideally we would try to balance runtime so that for example: * Fast (0.1s-1s) - Enough for allocation/instruction counts or rough runtime measurements. * normal (1s-2s) - Decent runtime measurements * slow (5s-10s) - More precision. Currently runtimes for slow just in shootout vary from 0.5s to 40s. So if we only run these two benchmarks we split benchmark time <2%:>98%. Not exactly a good use of resources. One side effect of adjusting runtimes would be that it would invalidate any long term performance comparison. So I'm also not sure if such a patch would be uncontroversial. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15357#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler