Reliably preventing let-floating and thunk overwriting?
The criterion benchmarking package is pretty nice to use in general, but it has a property that sticks in my craw: ensuring that pure code actually gets evaluated on every iteration through the benchmarking loop is awkward and fragile. My current scheme is to increment an Int on every iteration and pass that to the function being benchmarked. I document the need for the function to somehow use this Int, lest it be let-floated out to the top level, where it will be evaluated just once and our measurements will become nonsensical. I wonder if anyone else has any clever ideas about approaches that might be less intrusive in the API.
On 06/11/2009 05:39, Bryan O'Sullivan wrote:
The criterion benchmarking package is pretty nice to use in general, but it has a property that sticks in my craw: ensuring that pure code actually gets evaluated on every iteration through the benchmarking loop is awkward and fragile.
My current scheme is to increment an Int on every iteration and pass that to the function being benchmarked. I document the need for the function to somehow use this Int, lest it be let-floated out to the top level, where it will be evaluated just once and our measurements will become nonsensical.
I wonder if anyone else has any clever ideas about approaches that might be less intrusive in the API.
One reliable way to do this would be to dynamically load the code to be tested using the GHC API, and call this between each evaluation: foreign import ccall "revertCAFs" revertCAFs :: IO () (note revertCAFs only works on CAFs in dynamically-loaded object code, not interpreted or statically linked CAFs). Cheers, Simon
On Fri, Nov 6, 2009 at 1:03 AM, Simon Marlow
One reliable way to do this would be to dynamically load the code to be tested using the GHC API, and call this between each evaluation:
foreign import ccall "revertCAFs" revertCAFs :: IO ()
(note revertCAFs only works on CAFs in dynamically-loaded object code, not interpreted or statically linked CAFs).
Good to know, thanks. I ended up doing this, which is simpler and seems to work (at least for now): http://hpaste.org/fastcgi/hpaste.fcgi/view?id=11887
participants (2)
-
Bryan O'Sullivan -
Simon Marlow