On Fri, Feb 3, 2012 at 4:40 AM, Andres Löh wrote:

...

The way to handle tests for named packages would be to add a PackageConstraint for tests the way is already done for flags, but this is a lot more work because it would require getting deep into the dependency solver. However, it would make it easy to set the config flags during build for explicit targets so we could implement a test auto-run feature.

When I rewrote the solver, I was actually surprised that test flags are sort of bypassing the solver. I don't think it would be all that difficult to build it into the new solver properly, *if* I'd know what properly means. So if you can specify a reasonable behaviour you'd expect from the solver, I can try to implement it.

I looked at the top-down solver again to see if we could enable/disable tests and benchmarks through a constraint (the way flag assignments are done). To my surprise, it's not nearly as confusing as I thought it was the last time I looked at it (two years ago (!) during GSoC when I wrote that awful solver-bypassing test code). The attached patches do what I think we want. Test dependencies for targetted packages (named and source alike) are detected and installed, but '--enable-tests' is not propagated to dependencies. I only checked the top-down solver because I'm still not familiar with the modular solver. I also included a patch to automatically run the test suites when 'cabal install --enable-tests' is invoked and abort installation if the tests fail. I think this is a sensible default; if a user is going to ignore the results of the test suites, they probably don't want '--enable-tests'. Johan and Antoine: I tested my patch with some source packages I have around and the packages I could find on Hackage that have test suites, but that only assures me that '--enable-tests' does what _I_ think it should do. It sounds to me like you use this feature on a daily basis: could you try out this set of patches and let me know if this behavior meets your needs? Andres: I had to make a few changes to the modular solver to get cabal-install to compile with the new constraints, but I didn't go far enough to actually have the solver pull in the dependencies. I recorded these changes as a separate patch to make the changes more obvious. You understand the solvers better than I do; if I'm taking the wrong approach, please let me know. Otherwise, this is what I did outside the solvers, if you want to implement test and benchmark dependencies for the modular solver: 1. There is a new type, 'OptionalStanza', with two constructors indicating respectively that '--enable-tests' or '--enable-benchmarks' was specified. 2. Constraints are passed to the solver in the same way as flag assignments. The names of the target packages are determined, and a list [OptionalStanza] is associated with each. If a constructor is present in the list, that type of stanza is enabled. If the list is empty, neither tests nor benchmarks are enabled. When a solver is building an InstallPlan, it just needs to configure each package respecting the constraints to get the correct list of dependencies. Packages that aren't specified as targets will have no constraints, so they should be built without '--enable-tests' and '--enable-benchmarks'. Hopefully we can put this long-standing bug to rest! Thanks! -- Thomas Tuegel