On Mon, 2010-03-08 at 15:37 +0100, Marc Weber wrote:

Hi Maciej,

that's why I started hack-nix.

You can patch dependencies easily.

However you have to install the Nix package manager.

It also works with lates versions only because the dependency solves is written in Nix itself.

Which package is causing trouble to you?

For example happstack.

We can't expect package maintainers to test everything. So it must be people like you and me who fixes those changes.

Well. Except that it require bumping versions. Which according to hackage policy requires to fork a project (which makes it pointless in the first place)

The way to go is make hackage allow changing constraints on the fly notifying the author that he can update his repository.

This will work in most cases.

Bumping versions because a dependency has changed is bad as well.

This will cause to much overhead (and it dosen't solve the problem because the old package is still wrong).

Specifying dependencies must be decoupled from bumping versions.

It's because dep specs do depend on the "world" which can change..

At least that's what I think.

Hmm. When I was returning home I thought about some wiki-like system that would allow to say 'Package X is compatible/not compatible with Y'. Possibly something like: - Only the 'sure' deps are installed in default mode - When in 'expert' mode I can install any package which has not been marked as incompatible Then I can say that I tested built and: - It failed to built - It failed the automatic tests (if they exists)/does not work - It success So if there is versions: 0.7 0.8 0.9 1.0 1.0.1 1.1 1.2 1.3 1.4 And: - 0.8 failed to built - >=1.0 <1.1 was marked by author - 1.1 was marked as success - 1.3 failed to build Then - In default/normal mode it can be used with 1.0, 1.0.1 and 1.1 - In expert mode 0.9 and 1.2 can be installed in addition to above - Any version can be installed in 'I'm feeling lucky' mode when I explicitly say package to ignore some restriction Possibly it is needed to collect user karma (or possibly already account verification is sufficient).

If you're interested in Nix and hack-nix I can show you how everything works using an SSH session.

Ekhm... SSH?

Marc Weber

Regards

On Thu, Mar 11, 2010 at 08:54:15AM -0800, Rogan Creswick wrote:

(c) fix the unpacked version, and increment the version number by adding a new level of detail (so, 3.1.0 becomes 3.1.0.1). This version number never leaves my system -- it only exists to keep my cabal/ghc-pkg consistent!

A decent approach, except if the library in question is a core package to the compiler or a library that those packages depend on. I suffered some vast destruction doing the above to the Win32 package. Apparently packages like directory depends on this, breaking GHC irrecoverably if you try to propagate fixes. I managed to limp along for a bit with constraints, but ended up wiping the whole install and installing a clean one.

* Build reporting in the hackage server The idea here is that cabal sends back anonymous reports to the server to say if a package compiled or not, and against what versions of dependencies. This would make it clearer to maintainers if their dependency versions are correct. Additionally I think it would be useful for hackage to provide tweaked .cabal files for packages with updated constraints, even without new version uploads. We are proposing a GSoC project which would cover some of this.

Hmm. I guess there are 2 problems: - Privacy problem. I don't want the software to call home with data without asking. - Spoofing problem. Someone can feed wrong data easily without chance of being discovered (for example to prove how grate 'haxor' he is). Regards

Excerpts from Daniel Fischer's message of Sun Apr 11 16:20:30 -0400 2010:

I think in the default cabal config, build-reporting would be turned off and you'd have to turn it on yourself. Turning it on by default wouldn't be sensible.

I doubt you'd get very much runtime with that. I'd suggest prompting the user to submit a failed build report if the build fails. Cheers, Edward

Excerpts from Ivan Lazar Miljenovic's message of Sun Apr 11 17:37:15 -0400 2010:

"Edward Z. Yang" writes:

...

I doubt you'd get very much runtime with that. I'd suggest prompting the user to submit a failed build report if the build fails.

Exactly like how Windows keeps prompting you to allow it to send an error report to Microsoft? I don't know about you, but I always found it irritating...

I think the primary irritation is that Microsoft gets all of these error reports and they disappear into the great black VOID. Some public statistics might help alleviate the irritation. Cheers, Edward

Heinrich Apfelmus writes:

Perhaps exactly when the user is prompted?

"Would you like to send an anonymous report of this build failure to hackage.org? Users have been uploading 189 reports so far; yours would be the 190th report that ensures high quality Haskell packages! [y/n]"

Which, in its own way, means that cabal-install is still phoning home; you then even need network access if you're building packages that you've already downloaded the sources for. -- Ivan Lazar Miljenovic Ivan.Miljenovic@gmail.com IvanMiljenovic.wordpress.com

On Sun, 2010-04-11 at 18:43 +0200, Maciej Piechotka wrote:

...

* Build reporting in the hackage server The idea here is that cabal sends back anonymous reports to the server to say if a package compiled or not, and against what versions of dependencies. This would make it clearer to maintainers if their dependency versions are correct. Additionally I think it would be useful for hackage to provide tweaked .cabal files for packages with updated constraints, even without new version uploads. We are proposing a GSoC project which would cover some of this.

Hmm. I guess there are 2 problems:

- Privacy problem. I don't want the software to call home with data without asking.

Obviously it is important that the data be anonymous and that we do not send stuff without the user's knowledge. While there is not any directly identifying information in the existing anonymous build reports, one has to be very careful with how much access the server provides to the reports or it may become possible to infer identifying information. For example you might happen to know that I'm one of the few users on Sparc/Linux. Another possible attack is to try to discover timing of uploaded reports by the attacker inserting their own reports at regular intervals. If the server provided direct access to reports, in order of upload time, then this would give away timing of uploads. Similarly, if all reports were kept together then one could identify bunches of packages uploaded at the same time and infer they are from the same user. So yes, it needs careful thought. Perhaps we cannot give clients access to the raw reports at all. Perhaps it's enough to separate reports by package and to delay publishing new ones for a day or two, and to randomise the order when they are made available.

- Spoofing problem. Someone can feed wrong data easily without chance of being discovered (for example to prove how grate 'haxor' he is).

I don't think that's a major problem. One has to treat the data with that possibility in mind. There's also the option for authenticated non-anoymous reports with build logs. Duncan

On Sun, Apr 11, 2010 at 9:26 AM, Duncan Coutts

wrote:

On Mon, 2010-03-08 at 13:33 +0000, Maciej Piechotka wrote:

...

While I love Haskell it's packaging system have some problems - especially with parsec.

Currently I'm not able to install an practically anything using cabal due to version mismatches (or at least packages linking to both version of parsec).

I found the following problems in various cabal packages (the examples are only examples - please do not be offended): - The constraints are too loose. It is written that package works with '>0.6 && <1' but in 0.8 the API has been changed (example of tagsoup & hxt. Fixed hxt versions depends on ghc 6.12... and ghc 6.10 if you change 2 lines in cabal file) - The constraints are too tight. It is written that package works with 'parsec <3' but it can run with 3.0 and 3.1 (a lot of packages)

Here are a few things which I would like to see implemented that would help all this:

* Build reporting in the hackage server The idea here is that cabal sends back anonymous reports to the server to say if a package compiled or not, and against what versions of dependencies. This would make it clearer to maintainers if their dependency versions are correct. Additionally I think it would be useful for hackage to provide tweaked .cabal files for packages with updated constraints, even without new version uploads. We are proposing a GSoC project which would cover some of this.

* A package API tool and greater use of the PVP We do have a package versioning policy that maintainers can choose to follow. This helps users of the package write more accurate dependency version constraints. What we lack is a tool to help maintainers check that they are correctly following the PVP. We also have a GSoC proposal for a package tool.

* Private "build-depends" This would help the parsec 2 vs parsec 3 issue. Most packages that depend on parsec, or QuickCheck, do not export (as part of their public API) functions that use types from the parsec/QC packages. That is, the use of those packages is entirely encapsulated and invisible to clients. In this case, diamond dependency problems are impossible and it would be ok to use different versions of the same package within the same dependency graph. Currently cabal does not know which build-depends are public and which are private. The extension would be to let package authors mark some depends as private (though this would also need to be checked).

* Improved dependency resolution algorithm in cabal-install The constraint solver is currently not very smart. It does not do so well with tight version constraints, which is exactly what we will get more of, as people use the PVP more. The resolver could be improved to do significantly better in these situations. I have an algorithm in my head, but not much time to implement it.

I've noticed another type of diamond dependency problem. Suppose I build and install Foo today and it depends on Bar 2.0.0. In a week, I install Bar 2.0.1. Next I installed Baz that also depends on Bar, and it gets Bar 2.0.1. When I install a package that depends on Foo, Baz, and possibly Bar, then cabal won't be able to figure out proper dependencies because Foo needs one version of Bar and Baz needs a different version. But if all these packages followed the PVP, then either version of Bar could have been used, if it was used consistently. i'm not sure what is the best solution to this problem. Perhaps in this scenario, cabal should ask the user if it should rebuild Foo and/or Baz so they use the same Bar. I think this could lead to a lot of rebuilding, and that rebuilding gets tedious when you're doing it manually. Jason

On Mon, Apr 12, 2010 at 4:00 AM, Duncan Coutts

wrote:

On Sun, 2010-04-11 at 14:28 -0700, Jason Dagit wrote:

...

I've noticed another type of diamond dependency problem. Suppose I build and install Foo today and it depends on Bar 2.0.0. In a week, I install Bar 2.0.1. Next I installed Baz that also depends on Bar, and it gets Bar 2.0.1. When I install a package that depends on Foo, Baz, and possibly Bar, then cabal won't be able to figure out proper dependencies because Foo needs one version of Bar and Baz needs a different version.

That's exactly the problem that cabal-install solves. It will rebuild one or the other of Foo and Baz so that the dependency on Bar is consistent.

Weird. This has not been my experience. Jason

Ketil Malde wrote:

Perhaps it would also be possible to suggest library upgrades likely to remedy the problem in case of a build failure?

+1 for good error messages. +2 for "should I try upgrading libfoo? [yn]" integration (if configurable as AlwaysYes, AlwaysAsk, or AlwaysNo). -- Live well, ~wren