I thought about this but discarded it quite early on for the hacky nature. But that definitely looks like my best shot, the more I think about it. Thanks. 

On Thu, Jun 11, 2015 at 1:48 PM, Roman Cheplyaka <roma@ro-che.info> wrote:
In your CI script, you can rename one of your versions (say, mypkg to
mypkg-new) and then use -XPackageImports.

On 11/06/15 14:41, Alp Mestanogullari wrote:
> Hi -cafe,
>
> While we can easily benchmark different functions or libraries easily
> with criterion, I can't think of a reasonably easy (and accurate!) way
> of benchmarking two versions of the same package. And not necessarily
> version as in cabal version -- one of the use cases I have in mind would
> be running a benchmark suite whenever a PR gets merged to the main
> branch of a library, so the benchmark would need to compare
> the performance of the library's-code-before-merging and after.
>
> This definitely can't be accomplished with something like criterion
> because we can't have two different instances of a package in scope for
> a module, even with -XPackageImports.
>
> If we separately build the same program against two instances of the
> same library and run the benchmarks separately, this might happen far
> apart enough that the machine running this might be under a different
> load. This does however seem to be the only actual solution? Run
> separately at two different commits, diff the numbers, report.
>
> I have felt the need for a solution to this for quite some time, but
> have never needed it bad enough that it became a priority. Thinking
> about it again today, I thought I should drop an email to the list and
> see if fellow haskellers have an easy-to-use solution for this? I may
> very well be overlooking something.
>
> --
> Alp Mestanogullari
>
>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>





--
Alp Mestanogullari