| Does ghc-pkg allow to have two different versions of yesod-platform-2.7 | being installed? Not at the moment. But it should! S | -----Original Message----- | From: Henning Thielemann [mailto:lemming@henning-thielemann.de] | Sent: 22 November 2012 09:18 | To: Simon Peyton-Jones | Cc: Haskell Libraries (libraries@haskell.org); cabal-devel@haskell.org; | Michael Snoyman (michael@fpcomplete.com) | Subject: Re: Cabal and GHC | | | On Thu, 22 Nov 2012, Simon Peyton-Jones wrote: | | > The solution is obvious: we should make it possible to instally | > yesod-platform-2.7 twice, once version depending on data-default-0.4 | > once version depending on data-default-0.5. |

| > Now, if Cabal can figure out a plan based on an empty database, it can | deliver on that plan even in a non-empty database, without messing up | any existing installations. | | This is an interesting invariant, weaker than the more obvious one: | "Cabal should do the same plan no matter what the state of the database | is" which is in tension with "don't install more than you have to." Well I didn't say it should *ignore* the database. The current database might influence its plan. For example if, after installing both yesod-platform and Fay, Cabal is asked to install a package P which depends on yesod-platform data-default 0.4-0.8, then Cabal could pick the already-installed version of yesod-platform depending on data-default 0.5, rather than installing yet another version depending on data-default-0.8. So the invariant I suggest is * If it'd work in an empty database, it should work in any non-empty one * Installing X should never break the existing installation of Y Simon

Simon Peyton-Jones :

Well I didn't say it should *ignore* the database. The current database might influence its plan. For example if, after installing both yesod-platform and Fay, Cabal is asked to install a package P which depends on yesod-platform data-default 0.4-0.8, then Cabal could pick the already-installed version of yesod-platform depending on data-default 0.5, rather than installing yet another version depending on data-default-0.8.

So the invariant I suggest is * If it'd work in an empty database, it should work in any non-empty one * Installing X should never break the existing installation of Y

This makes a lot of sense to me. I think it is also time to admit that we are not the first trying to solve this problem. For example, Scala has the sbt tool: http://www.scala-sbt.org I am sure, it is not perfect, but it seems much more flexible and versatile than Cabal. Interestingly, the Scala guys were smart enough to avoid re-inventing the wheel and are using Apchae's Ivy http://ant.apache.org/ivy/ to do their dependency management. Manuel

"Cabal should do the same plan no matter what the state of the database is" which is in tension with "don't install more than you have to."

This I like, except instead of "cabal", it should be something new. With a combination of curators and a synchronized release schedule, I see a system like this running very smoothly, and you could even bridge the gap between both ideals. A social solution will be much more effective than a technical solution. I still agree with Michael's post that it would be best to let cabal keep doing its thing, and instead create a new, more convenient tool on top of it with this added social layer of curation and synchronization. Perhaps the best solution would be to add the functionality Simon suggests, as well as adopting the hopelessly complex Nix-style package management (referring to Brandon's comments), and then giving sense to it with a social layer on top. -- Dan Burton

Having GHC support multiple copies of the same package/version would certainly be a step in the right direction. It still wouldn't completely address the goals I'm getting at in my blog post. What I'm really aiming for overall is to provide users with a stable, vetted set of packages that are known to work well together. But for those of us who will still be using plain Hackage, this improvement would be very helpful. Indeed. I've always thought of the Haskell Platform as doing the "stable set" part, though there must be plenty of space between HP at one end and Hackage at the other, and that's what you are exploring I think. Regardless, being able to install multiple instantiations of the same package/version is desirable for the "stable set" story, so that more than one stable set is possible; and so that trying to use a package outside the stable set doesn't mess up your stable-set installation. The question is: who will do the work? Simon From: michael.snoyman@gmail.com [mailto:michael.snoyman@gmail.com] On Behalf Of Michael Snoyman Sent: 22 November 2012 10:40 To: Simon Peyton-Jones Cc: Haskell Libraries (libraries@haskell.org); cabal-devel@haskell.org Subject: Re: Cabal and GHC On Thu, Nov 22, 2012 at 10:32 AM, Simon Peyton-Jones mailto:simonpj@microsoft.com> wrote: Michael Snoyman wrote a blog post about Solving Cabal Hall, which came across via the peerless Haskell Weekly News. http://www.yesodweb.com/blog/2012/11/solving-cabal-hell But I'd be unlikely to return there so I'm posting this to the libraries and cabal-devel lists. I'm not deep in Cabal lore, but it seems to me that there is a fairly easy way to do a Lot Better than we are now. Let's go back to "Identifying the Problem". Michael doesn't say, but I'm guessing that Fay depends on yesod-platform. Very well, so Cabal sees I'm installing Fay, which depends on yesod-platform, which dpends on data-default and depends directly on data-default If the package database was empty, Cabal would try to figure out a version of data-default that is acceptable to both yesod-platform and to Fay. And that's what we want! Suppose that Cabal figures out that yesod-platform-2.7 and data-default-0,4 would work. Then, in an empty package database, it could just go ahead and install those. But in your example, the database isn't empty; we have already isntalled yesod-platform-2.7, dependin gon data-default-0.5. And (here's the rub) you can only have yesod-platform-2.7 installed once. One solution might be 1. *un* install yesod-platform-2.7 (depending on data-default-0.5) and 2. *re* install it (depending on data-default-0.4). But that would break any programs that use both yesod-platform-2.7 and data-default-0.5. The solution is obvious: we should make it possible to instally yesod-platform-2.7 twice, once version depending on data-default-0.4 once version depending on data-default-0.5. Now, if Cabal can figure out a plan based on an empty database, it can deliver on that plan even in a non-empty database, without messing up any existing installations. Of course, if Fay depends exclusively on data-default-0.4, and yesod-platform depends exclusively on data-default-0.5, then they genuinely are incompatible, and Cabal can and should say so. That's not Cabal Hell. That's just saying that the package authors have *specified* that they are incompatible. Summary. I may be way off beam here, but I think we can easily make things way better than they are. By "easily" I mean with just design and implementation work. That's not free, and no one has any time.... but it's well within reach. I'd even been wondering about trying to crowd-fund this development so that Well Typed can do it. But let's first see if it'd solve the problem. There must be a wiki page somewhere that articulates the challenge, points to blog posts about it, and discusses solutions. Simon _______________________________________________ Libraries mailing list Libraries@haskell.orgmailto:Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries Hi Simon, Having GHC support multiple copies of the same package/version would certainly be a step in the right direction. It still wouldn't completely address the goals I'm getting at in my blog post. What I'm really aiming for overall is to provide users with a stable, vetted set of packages that are known to work well together. But for those of us who will still be using plain Hackage, this improvement would be very helpful. MIchael

On 22 November 2012 10:39, Michael Snoyman wrote:

On Thu, Nov 22, 2012 at 10:32 AM, Simon Peyton-Jones wrote:

...

Michael Snoyman wrote a blog post about Solving Cabal Hall, which came across via the peerless Haskell Weekly News. http://www.yesodweb.com/blog/2012/11/solving-cabal-hell

...

The solution is obvious: we should make it possible to instally yesod-platform-2.7 twice,

Having GHC support multiple copies of the same package/version would certainly be a step in the right direction. It still wouldn't completely address the goals I'm getting at in my blog post. What I'm really aiming for overall is to provide users with a stable, vetted set of packages that are known to work well together. But for those of us who will still be using plain Hackage, this improvement would be very helpful.

Right, and we need both. Your suggestion is very sensible. It's similar to the suggestion people have made before about having "testing" and "stable" sets of packages (these suggestions often come from people using debian terminology), and managing things like a distribution. In both cases the point is to provide something to end users where we've set up the available packages such that we know most things work together. (This is why debian users don't run into problems most of the time, because volunteers have done a huge amount of work behind the scenes to eliminate conflicting versions). So yes we need that, and it will require a fair bit of work, and we need to distribute that amongst many people. I think eventually the best mechanism to do it will be to use hackage and use tagging, but we can certainly experiment now by using separate hackage instances. But we also need to address the problems of developers in the thick of things, who must work with the latest and greatest unsable versions, before people have got round to smoothing out all the conflicting dependencies and patching packages so they can all use the same versions of common dependencies. That's where the nix-style approach that we've been advocating for years comes in. That's what Philipp Schuster's GSoC this summer was all about. That's aiming to do exactly what Simon is talking about here. It's about allowing sets of packages to be installed simultaneously that have inconsistent sets of dependencies. There's a slight overlap with sandboxing, but the way I see it, the nix approach is the right underlying implementation mechanism and then sandboxing is more of a UI issue. Duncan

On Thu, Nov 22, 2012 at 8:41 PM, Henning Thielemann wrote:

On Thu, 22 Nov 2012, Brandon Allbery wrote:

...

I am wondering if we're trying to solve the problem in the wrong way. The core of the problem is that various things get baked into libraries in the name of optimization, such that a given binary library has dependencies on precise versions of other libraries; it's not like C where anything supporting the ABI can use the same .a/.lib or .so/.dll/.dylib.

C supports inlining. This should cause the same kind of problems, shouldn't it?

Indeed, and in Qt for example which has a strict binary compatibility guarantee, making a public function inline means you effectively cannot change it until the next major release. (In this case "making it inline" and "including the definition in the header file" are effectively synonymous, one requires the other.) The problem is that in Haskell inlining is a lot more important for performance.

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

-- Your ship was destroyed in a monadic eruption.

On 23 November 2012 21:06, Simon Marlow wrote:

On 22/11/2012 19:54, Gábor Lehel wrote:

...

On Thu, Nov 22, 2012 at 8:41 PM, Henning Thielemann wrote:

...

On Thu, 22 Nov 2012, Brandon Allbery wrote:

...

I am wondering if we're trying to solve the problem in the wrong way. The core of the problem is that various things get baked into libraries in the name of optimization, such that a given binary library has dependencies on precise versions of other libraries; it's not like C where anything supporting the ABI can use the same .a/.lib or .so/.dll/.dylib.

Right, I tend to agree. Multiple installs of packages make me feel uncomfortable and sleep less well at night... :)

I would like to see GHC support fixed ABIs, and the work I did with ABI hashing in GHC was aiming towards exactly that.

For fixed ABIs you would need to have the user explicitly declare every inline function, and then hash the definitions as part of the ABI. (also do something about strictness and arity, and other cross-module optimisation hints). It might be painful, but it could be optional, and the gains are quite nice: the ability to upgrade a library in-place without recompiling everything that depends on it. Especially now that we're moving towards shared libraries, this would become more useful.

Yes that would be really wonderful!! The current ABI situation is really very painful for packagers and beyond IMHO. To me it is actually the largest current problem with ghc. Jens