Cabal and simultaneous installations of the same package

Simon Peyton Jones

23 Mar 2015 23 Mar '15

4:45 a.m.

Dear Cabal developers You'll probably have seen the thread about the Haskell Platform. Among other things, this point arose: | Another thing we should fix is the (now false) impression that HP gets in | the way of installing other packages and versions due to cabal hell. People mean different things by "cabal hell", but the inability to simultaneously install multiple versions of the same package, compiled against different dependencies is certainly one of them, and I think it is the one that Yitzchak is referring to here. But some time now GHC has allowed multiple versions of the same package (compiled against different dependencies) to be installed simultaneously. So all we need to do is to fix Cabal to allow it too, and thereby kill of a huge class of cabal-hell problems at one blow. But time has passed and it hasn't happened. Is this because I'm misunderstanding? Or because it is harder than I think? Or because there are much bigger problems? Or because there is insufficient effort available? Or what? Unless I'm way off beam, this "multiple installations of the same package" thing has been a huge pain forever, and the solution is within our grasp. What's stopping us grasping it? Simon

Show replies by date

Michael Snoyman

23 Mar 23 Mar

4:52 a.m.

I'm in favor of adding support to Cabal to allow for this situation. However: I highly doubt this will be the panacea as predicted. It's already a huge source of confusion for people using GHCi what they get messages about "ByteString is not ByteString." In fact, this confusion is so prevalent that I wrote a blog post about it in September[1]. I strongly believe that the best end-user experience comes from having a single mapping from package name to actually installed package/version/dependencies. I'd go even farther and say that users would be well served by having a single mapping from module name to installed module. In fact, Adam Bergmark even created an issue[2] to look into adding support to Stackage to start enforcing this situation, though the response to a blog post I wrote on that[3] implies that people are not so interested in addressing that problem. So my word of warning here is: if we simply throw multiple package/version/dependency combinations at users via cabal, there's a good chance that we'll do more harm than good. [1] http://www.yesodweb.com/blog/2014/09/woes-multiple-package-versions [2] https://github.com/fpco/stackage/issues/416 [3] http://www.yesodweb.com/blog/2014/02/module-name-conflicts On Mon, Mar 23, 2015 at 10:45 AM Simon Peyton Jones wrote:

...

Dear Cabal developers

You'll probably have seen the thread about the Haskell Platform.

Among other things, this point arose:

| Another thing we should fix is the (now false) impression that HP gets in | the way of installing other packages and versions due to cabal hell.

People mean different things by "cabal hell", but the inability to simultaneously install multiple versions of the same package, compiled against different dependencies is certainly one of them, and I think it is the one that Yitzchak is referring to here.

But some time now GHC has allowed multiple versions of the same package (compiled against different dependencies) to be installed simultaneously. So all we need to do is to fix Cabal to allow it too, and thereby kill of a huge class of cabal-hell problems at one blow.

But time has passed and it hasn't happened. Is this because I'm misunderstanding? Or because it is harder than I think? Or because there are much bigger problems? Or because there is insufficient effort available? Or what?

Unless I'm way off beam, this "multiple installations of the same package" thing has been a huge pain forever, and the solution is within our grasp. What's stopping us grasping it?

Simon

_______________________________________________ Libraries mailing list Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

Herbert Valerio Riedel

5:17 a.m.

On 2015-03-23 at 09:52:31 +0100, Michael Snoyman wrote: [...]

...

So my word of warning here is: if we simply throw multiple package/version/dependency combinations at users via cabal, there's a good chance that we'll do more harm than good.

Nevertheless, the best way to find out is to actually try it by implementing it... if in fact it doesn't work out as well as expected, we don't have to ship it :-)

Simon Peyton Jones

5:52 a.m.

It's already a huge source of confusion for people using GHCi what they get messages about "ByteString is not ByteString." Reading your blog post [1] it seems that we are addressing different questions: · My proposal is only that the act of *installing* a package does not break existing installed package, and is not rejected because it risks doing so. · You agree that the confusing behaviour you describe can’t happen with Cabal. In any one build, Cabal can ensure that only one version of each package is used in the build, so such a message could never show up. · What you want is for the confusing behaviour to be true of GHCi too. Well that’s simple enough: ensure that the set of exposed packages (ie the ones you say ‘import M’ for), is consistent in the same way. The point is that I may need to install a bunch of packages to build a program. If I’m using Cabal, none of those newly installed packages need be exposed; I simply need them there so I can compile my program (using Cabal). But at the moment I can’t do that. That leaves open the following question. Suppose · I want to install and expose package P and Q · But alas, P depends on containers 2.9 and Q depends on containers 3.1 Now I’m stuck. But there is a good reason for being stuck, and one that is explicable. Simon From: Michael Snoyman [mailto:michael@snoyman.com] Sent: 23 March 2015 08:53 To: Simon Peyton Jones; cabal-devel@haskell.org Cc: haskell-platform@projects.haskell.org; haskell-infrastructure@community.galois.com; Haskell Libraries; ghc-devs@haskell.org Subject: Re: Cabal and simultaneous installations of the same package I'm in favor of adding support to Cabal to allow for this situation. However: I highly doubt this will be the panacea as predicted. It's already a huge source of confusion for people using GHCi what they get messages about "ByteString is not ByteString." In fact, this confusion is so prevalent that I wrote a blog post about it in September[1]. I strongly believe that the best end-user experience comes from having a single mapping from package name to actually installed package/version/dependencies. I'd go even farther and say that users would be well served by having a single mapping from module name to installed module. In fact, Adam Bergmark even created an issue[2] to look into adding support to Stackage to start enforcing this situation, though the response to a blog post I wrote on that[3] implies that people are not so interested in addressing that problem. So my word of warning here is: if we simply throw multiple package/version/dependency combinations at users via cabal, there's a good chance that we'll do more harm than good. [1] http://www.yesodweb.com/blog/2014/09/woes-multiple-package-versions [2] https://github.com/fpco/stackage/issues/416 [3] http://www.yesodweb.com/blog/2014/02/module-name-conflicts On Mon, Mar 23, 2015 at 10:45 AM Simon Peyton Jones mailto:simonpj@microsoft.com> wrote: Dear Cabal developers You'll probably have seen the thread about the Haskell Platform. Among other things, this point arose: | Another thing we should fix is the (now false) impression that HP gets in | the way of installing other packages and versions due to cabal hell. People mean different things by "cabal hell", but the inability to simultaneously install multiple versions of the same package, compiled against different dependencies is certainly one of them, and I think it is the one that Yitzchak is referring to here. But some time now GHC has allowed multiple versions of the same package (compiled against different dependencies) to be installed simultaneously. So all we need to do is to fix Cabal to allow it too, and thereby kill of a huge class of cabal-hell problems at one blow. But time has passed and it hasn't happened. Is this because I'm misunderstanding? Or because it is harder than I think? Or because there are much bigger problems? Or because there is insufficient effort available? Or what? Unless I'm way off beam, this "multiple installations of the same package" thing has been a huge pain forever, and the solution is within our grasp. What's stopping us grasping it? Simon _______________________________________________ Libraries mailing list Libraries@haskell.orgmailto:Libraries@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

Michael Snoyman

5:58 a.m.

On Mon, Mar 23, 2015 at 11:53 AM Simon Peyton Jones wrote:

...

It's already a huge source of confusion for people using GHCi what they get messages about "ByteString is not ByteString."

Reading your blog post [1] it seems that we are addressing different questions:

· My proposal is only that the act of **installing** a package does not break existing installed package, and is not rejected because it risks doing so.

Thank you for the clarification, I had misread that. On that front: I agree.

...

· You agree that the confusing behaviour you describe can’t happen with Cabal. In any one build, Cabal can ensure that only one version of each package is used in the build, so such a message could never show up.

I've seen people discussing exactly such a change to Cabal's behavior, so I mistakenly took your comments to be heading in that direction. While I think there *might* be some future where we could expose that functionality, it could be incredibly confusing. I'd feel much better starting off with simply the act of installing.

...

· What you want is for the confusing behaviour to be true of GHCi too. Well that’s simple enough: ensure that the set of *exposed* packages (ie the ones you say ‘import M’ for), is consistent in the same way. The point is that I may need to install a bunch of packages to build a program. If I’m using Cabal, none of those newly installed packages need be exposed; I simply need them there so I can compile my program (using Cabal). But at the moment I can’t do that.

That leaves open the following question. Suppose

· I want to install *and expose* package P and Q

· But alas, P depends on containers 2.9 and Q depends on containers 3.1

Now I’m stuck. But there is a good reason for being stuck, and one that is explicable.

...

Simon Peyton Jones

6:05 a.m.

If I'm reading this correctly, the proposal then would be to have cabal automatically hide packages (as oppose to unregister them) to arrive at a world where all exposed packages are consistent. Extrapolating for the case you mention above * if I installed P and then Q, I'd end up with containers-3.1 and Q exposed, and containers-2.9 and P hidden * if I installed Q and then P, I'd end up with containers-2.9 and P exposed, and containers-3.1 and Q hidden But either way, all four package/versions would be available, and cabal would be able to select an appropriate subset of packages when configuring. Does that sound about right? Correct, esp the bit that I have emboldened. For the former bullets, perhaps Cabal might ask you want you want to do. That still leaves open questions. What happens if you say “ghc –package P –package Q Foo.hs”? Should GHC complain that you’ve chosen an inconsistent set? Perhaps! (NB if P and Q use containers only internally, and do not expose any types from containers, then arguably it’s ok; but I’d argue for jumping that bridge when we come to it.) Simon From: Michael Snoyman [mailto:michael@snoyman.com] Sent: 23 March 2015 09:58 To: Simon Peyton Jones; cabal-devel@haskell.org Cc: haskell-platform@projects.haskell.org; haskell-infrastructure@community.galois.com; Haskell Libraries; ghc-devs@haskell.org Subject: Re: Cabal and simultaneous installations of the same package On Mon, Mar 23, 2015 at 11:53 AM Simon Peyton Jones mailto:simonpj@microsoft.com> wrote: It's already a huge source of confusion for people using GHCi what they get messages about "ByteString is not ByteString." Reading your blog post [1] it seems that we are addressing different questions: • My proposal is only that the act of *installing* a package does not break existing installed package, and is not rejected because it risks doing so. Thank you for the clarification, I had misread that. On that front: I agree. • You agree that the confusing behaviour you describe can’t happen with Cabal. In any one build, Cabal can ensure that only one version of each package is used in the build, so such a message could never show up. I've seen people discussing exactly such a change to Cabal's behavior, so I mistakenly took your comments to be heading in that direction. While I think there *might* be some future where we could expose that functionality, it could be incredibly confusing. I'd feel much better starting off with simply the act of installing. • What you want is for the confusing behaviour to be true of GHCi too. Well that’s simple enough: ensure that the set of exposed packages (ie the ones you say ‘import M’ for), is consistent in the same way. The point is that I may need to install a bunch of packages to build a program. If I’m using Cabal, none of those newly installed packages need be exposed; I simply need them there so I can compile my program (using Cabal). But at the moment I can’t do that. That leaves open the following question. Suppose • I want to install and expose package P and Q • But alas, P depends on containers 2.9 and Q depends on containers 3.1 Now I’m stuck. But there is a good reason for being stuck, and one that is explicable. If I'm reading this correctly, the proposal then would be to have cabal automatically hide packages (as oppose to unregister them) to arrive at a world where all exposed packages are consistent. Extrapolating for the case you mention above * if I installed P and then Q, I'd end up with containers-3.1 and Q exposed, and containers-2.9 and P hidden * if I installed Q and then P, I'd end up with containers-2.9 and P exposed, and containers-3.1 and Q hidden But either way, all four package/versions would be available, and cabal would be able to select an appropriate subset of packages when configuring. Does that sound about right? Michael

Miëtek Bak

11:28 a.m.

On 2015-03-23, at 09:52, Simon Peyton Jones wrote:

...

The point is that I may need to install a bunch of packages to build a program. If I’m using Cabal, none of those newly installed packages need be exposed; I simply need them there so I can compile my program (using Cabal). But at the moment I can’t do that.

Do you mean that you may need to install a bunch of packages to build a _build-tool_ (such as alex, happy, c2hs, cpps…), in order to compile your program? If yes, then one way to avoid having these packages pollute your build environment is building each Haskell program in a separate sandbox. There are some tools which attempt to simplify this process. Halcyon goes a bit further — Halcyon allows you to declare other Haskell programs to be installed as build-time (or run-time!) dependencies for your program. If needed, they will be built on-the-fly; otherwise, they will be restored from previously-built archives. This works around long-standing cabal-install issues: https://github.com/haskell/cabal/issues/220 https://github.com/haskell/cabal/issues/779 See the Haskell Language source code for an example: https://halcyon.sh/examples/#haskell-language See the Halcyon reference for details: https://halcyon.sh/reference/#halcyon_sandbox_extra_apps https://halcyon.sh/reference/#halcyon_extra_apps -- Miëtek https://mietek.io

Johan Tibell

11:05 a.m.

On Mon, Mar 23, 2015 at 9:45 AM, Simon Peyton Jones wrote:

...

But time has passed and it hasn't happened. Is this because I'm misunderstanding? Or because it is harder than I think? Or because there are much bigger problems? Or because there is insufficient effort available? Or what?

I have no idea what the status of this is or if GHC indeed has all the things we need. Perhaps Edward could comment.

Simon Peyton Jones

11:32 a.m.

I have no idea what the status of this is or if GHC indeed has all the things we need. Perhaps Edward could comment. I’m pretty confident that GHC has all the things needed, and has done since last summer. It’s Cabal that doesn’t! But as you say, Edward Y can confirm. Simon From: Johan Tibell [mailto:johan.tibell@gmail.com] Sent: 23 March 2015 15:05 To: Simon Peyton Jones Cc: cabal-devel@haskell.org; haskell-platform@projects.haskell.org; haskell-infrastructure@community.galois.com; Haskell Libraries; ghc-devs@haskell.org; Edward Yang Subject: Re: Cabal and simultaneous installations of the same package On Mon, Mar 23, 2015 at 9:45 AM, Simon Peyton Jones mailto:simonpj@microsoft.com> wrote: But time has passed and it hasn't happened. Is this because I'm misunderstanding? Or because it is harder than I think? Or because there are much bigger problems? Or because there is insufficient effort available? Or what? I have no idea what the status of this is or if GHC indeed has all the things we need. Perhaps Edward could comment.

Gershom B

1:31 p.m.

On Mon, Mar 23, 2015 at 4:45 AM, Simon Peyton Jones wrote:

...

| Another thing we should fix is the (now false) impression that HP gets in | the way of installing other packages and versions due to cabal hell.

People mean different things by "cabal hell", but the inability to simultaneously install multiple versions of the same package, compiled against different dependencies is certainly one of them, and I think it is the one that Yitzchak is referring to here.

My understanding is that the problem Yitz is discussing is much more straightforward and easy to fix. The issue is that if I have an "empty" global package db and enter a sandbox and cabal-install something, then I will get one package install plan. But if I have a "well populated" package db, then attempt to cabal-install something in the sandbox, the plan will attempt to use the packages from that global db, and so will often be a different, and perhaps worse plan (especially if it runs into quirks in the solver). This is discussed on the associated reddit thread here: http://www.reddit.com/r/haskell/comments/2zts44/wither_the_platform/cpmx9v7 As I proposed on the main email thread, I think it would make sense to make the easy, default option the one which, in sandboxes, effectively ignores the global db for solving. This leads to more duplication and longer build times, but it is safer. However, we should have an easy setting/flag (to be set when initializing a sandbox or any time thereafter) which can switch the preference over to preferring to use already installed versions in the global db. That way, new users can, naively, get a _lot_ of isolation with sandboxes, and those who know what they are doing can opt out in order to reduce duplication of installs and cut down on build times. Addition of a `prefer-global-db` flag and a good default setting for it (i.e., false) seems to me like it would resolve the _central_ issue people have with the platform, and situate us well to unilaterally recommend it as the default install option. Cheers, Gershom

Simon Peyton Jones

4:13 p.m.

But I'm hazy about why sandboxes are needed at all. As I understand it, they were invented to solve the very problem that is now solved (if only Cabal could take advantage of it). Simon ________________________________________ From: Gershom B Sent: 23 March 2015 17:31 To: Simon Peyton Jones Cc: cabal-devel@haskell.org; haskell-platform@projects.haskell.org; haskell-infrastructure@community.galois.com; Haskell Libraries; ghc-devs@haskell.org Subject: Re: Cabal and simultaneous installations of the same package On Mon, Mar 23, 2015 at 4:45 AM, Simon Peyton Jones wrote:

...

| Another thing we should fix is the (now false) impression that HP gets in | the way of installing other packages and versions due to cabal hell.

People mean different things by "cabal hell", but the inability to simultaneously install multiple versions of the same package, compiled against different dependencies is certainly one of them, and I think it is the one that Yitzchak is referring to here.

Duncan Coutts

6:10 p.m.

On Mon, 2015-03-23 at 20:13 +0000, Simon Peyton Jones wrote:

...

But I'm hazy about why sandboxes are needed at all. As I understand it, they were invented to solve the very problem that is now solved (if only Cabal could take advantage of it).

Yes, the nix approach would subsume sandboxes. I cover this in this post http://www.well-typed.com/blog/preview/how-we-might-abolish-cabal-hell-2/ Duncan

Duncan Coutts

6:33 p.m.

On Mon, 2015-03-23 at 22:10 +0000, Duncan Coutts wrote:

...

On Mon, 2015-03-23 at 20:13 +0000, Simon Peyton Jones wrote:

...
But I'm hazy about why sandboxes are needed at all. As I understand it, they were invented to solve the very problem that is now solved (if only Cabal could take advantage of it).

Yes, the nix approach would subsume sandboxes.

I cover this in this post http://www.well-typed.com/blog/preview/how-we-might-abolish-cabal-hell-2/

Oops, permanent link: http://www.well-typed.com/blog/2015/01/how-we-might-abolish-cabal-hell-part-...

Duncan Coutts

6:07 p.m.

On Mon, 2015-03-23 at 08:45 +0000, Simon Peyton Jones wrote:

...

Dear Cabal developers

You'll probably have seen the thread about the Haskell Platform.

Among other things, this point arose:

| Another thing we should fix is the (now false) impression that HP gets in | the way of installing other packages and versions due to cabal hell.

People mean different things by "cabal hell", but the inability to simultaneously install multiple versions of the same package, compiled against different dependencies is certainly one of them, and I think it is the one that Yitzchak is referring to here.

But some time now GHC has allowed multiple versions of the same package (compiled against different dependencies) to be installed simultaneously.

That's technically true of existing ghc versions, though ghc-pkg does not directly allow registering multiple instances of the same version of a package. As of 7.10 we can actually use ghc-pkg to register multiple instances, using ghc-pkg register --enable-multi-instance Also since 7.10 we can have "environment" files that say what packages to expose. There can be a per-user default one, or per-user named ones (which can be used via the ghc command line or via env var), or local environments in a directory. While Cabal has always built things with consistent deps and solved the problem of "which ByteString do I mean", the same has not been true for GHCi. With this new environment mechanism Cabal can create the environments and then when the user runs ghc/ghci then they get the set of packages previously set up by Cabal. Elsewhere in this thread you say:

...

What you want is for the confusing behaviour to be true of GHCi too. Well that’s simple enough: ensure that the set of exposed packages (ie the ones you say ‘import M’ for), is consistent in the same way. The point is that I may need to install a bunch of packages to build a program. If I’m using Cabal, none of those newly installed packages need be exposed; I simply need them there so I can compile my program (using Cabal). But at the moment I can’t do that.

Yes, that is exactly what these environment files are intended for. Myself and Edsko implemented these features (for the IHG) to enable future Cabal versions to take the multi instance approach fully.

...

So all we need to do is to fix Cabal to allow it too, and thereby kill of a huge class of cabal-hell problems at one blow.

With these features now implemented in GHC we're in a position to turn attention to Cabal to make use of them.

...

But time has passed and it hasn't happened. Is this because I'm misunderstanding? Or because it is harder than I think? Or because there are much bigger problems? Or because there is insufficient effort available? Or what?

There's a number of parts remaining to do.

...

Unless I'm way off beam, this "multiple installations of the same package" thing has been a huge pain forever, and the solution is within our grasp. What's stopping us grasping it?

Yes, we're closer than ever. I covered more of the details in this blog post: http://www.well-typed.com/blog/preview/how-we-might-abolish-cabal-hell-2/ So some of the remaining parts: Cabal needs to assign proper installed package ids, like nix does. These should be based on the hash of all inputs to a package build. In particular that means a hash of the content of the sources. This is easy for tarballs but a bit harder to do accurately for unpacked build trees. There are some new UI issues to deal with. The user interface will have to make this issue of multiple consistent environments be explicit in the user interface. We need to know what env we are in, what is in it, what other envs are available and be able to switch between them. Suppose that we enforce that each environment be fully consistent. Then when I "cabal install Q" and it cannot find a solution to install everything in the current environment plus the one extra package such that they all have consistent deps, then what should it do? Suppose that Q really could be installed on its own, but cannot be installed consistently with the other things in this env. Should it suggest that you make a new environment? There are some details to work out here so that we don't make a confusing UI. There's also an unresolved issue about when we try to reuse existing installed dependencies. One approach is to say that we make install plans without considering what is already available in the package store, and then only re-use existing ones if the installed package Ids happen to match up. The other approach is to say that the solver should actively try to reuse installed instances when it can. The latter is what we do now, to try and reduce the number of reinstalls. But when there are dozens of versions available this is harder: we need to know more information to know if it is safe to re-use an existing instance. (There are examples where it's clearly not safe to reuse packages.) Or a pragmatic approach might be to try and reuse existing installed instances within the current environment but not actively try to reuse other instances available in the package store. On a related topic, along with the London HUG we're trying to organise a couple infrastructure hackathons in London. The aim would be to work on Cabal/Hackage related things. The plan is to have two hackathons 6-8 weeks apart, to get new people set up with projects at the first and to get things merged in the second. For Cabal, this project would be my focus, trying to get people to work on different aspects of it. I gave a talk at the London HUG recently about getting involved with hacking on Cabal/Hackage. Duncan

Simon Peyton Jones

20 Apr 20 Apr

4:12 a.m.

Friends We started this thread (below) a month ago, but it has once more run out of steam. The last contribution was from Vishal Agrawal I am already planning to do a GSoC project based on it with a slightly larger aim. You can find my work in progress proposal at https://gist.github.com/fugyk/37510958b52589737274. Also I have written a patch to make cabal non-destructive at https://github.com/fugyk/cabal/commit/45ec5edbaada1fd063c67d6109e69efa0e732e.... Can you review the proposal and give me suggestions. I don't feel qualified to drive this process, but I do think it's important to complete it. (I might be wrong about this too... please say so if so.) Nor do I understand why it's difficult to tie up the bow; the underlying infrastructure work is done. Duncan especially: how can we make progress? Do you think it's important to make progress, or are other things in cabal-land more important? My reason for thinking that it's important is that it appears to be the root cause of many people's difficulties with Haskell and Cabal. It might not be a panacea for all ills; but it might be a cheap remedy for a significant proportion of ills. And that would be a Good Thing. Thanks Simon | -----Original Message----- | From: ghc-devs [mailto:ghc-devs-bounces@haskell.org] On Behalf Of Simon | Peyton Jones | Sent: 23 March 2015 08:46 | To: cabal-devel@haskell.org | Cc: haskell-platform@projects.haskell.org; haskell- | infrastructure@community.galois.com; Haskell Libraries; ghc- | devs@haskell.org | Subject: Cabal and simultaneous installations of the same package | | Dear Cabal developers | | You'll probably have seen the thread about the Haskell Platform. | | Among other things, this point arose: | | | Another thing we should fix is the (now false) impression that HP | | gets in the way of installing other packages and versions due to | cabal hell. | | People mean different things by "cabal hell", but the inability to | simultaneously install multiple versions of the same package, | compiled against different dependencies is certainly one of them, | and I think it is the one that Yitzchak is referring to here. | | But some time now GHC has allowed multiple versions of the same package | (compiled against different dependencies) to be installed | simultaneously. So all we need to do is to fix Cabal to allow it too, | and thereby kill of a huge class of cabal-hell problems at one blow. | | But time has passed and it hasn't happened. Is this because I'm | misunderstanding? Or because it is harder than I think? Or because | there are much bigger problems? Or because there is insufficient | effort available? Or what? | | Unless I'm way off beam, this "multiple installations of the same | package" thing has been a huge pain forever, and the solution is within | our grasp. What's stopping us grasping it? | | Simon | | _______________________________________________ | ghc-devs mailing list | ghc-devs@haskell.orgmailto:ghc-devs@haskell.org | http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Simon Peyton Jones

7 May 7 May

4:34 p.m.

Dear Cabal developers I guess everyone is busy, but I feel a bit stuck on knowing how to make progress on this thread. Thanks Simon From: Simon Peyton Jones Sent: 20 April 2015 09:12 To: Simon Peyton Jones; cabal-devel@haskell.org Cc: haskell-infrastructure@community.galois.com; Haskell Libraries; ghc-devs@haskell.org Subject: RE: Cabal and simultaneous installations of the same package Friends We started this thread (below) a month ago, but it has once more run out of steam. The last contribution was from Vishal Agrawal I am already planning to do a GSoC project based on it with a slightly larger aim. You can find my work in progress proposal at https://gist.github.com/fugyk/37510958b52589737274. Also I have written a patch to make cabal non-destructive at https://github.com/fugyk/cabal/commit/45ec5edbaada1fd063c67d6109e69efa0e732e.... Can you review the proposal and give me suggestions. I don't feel qualified to drive this process, but I do think it's important to complete it. (I might be wrong about this too... please say so if so.) Nor do I understand why it's difficult to tie up the bow; the underlying infrastructure work is done. Duncan especially: how can we make progress? Do you think it's important to make progress, or are other things in cabal-land more important? My reason for thinking that it's important is that it appears to be the root cause of many people's difficulties with Haskell and Cabal. It might not be a panacea for all ills; but it might be a cheap remedy for a significant proportion of ills. And that would be a Good Thing. Thanks Simon | -----Original Message----- | From: ghc-devs [mailto:ghc-devs-bounces@haskell.org] On Behalf Of Simon | Peyton Jones | Sent: 23 March 2015 08:46 | To: cabal-devel@haskell.orgmailto:cabal-devel@haskell.org | Cc: haskell-platform@projects.haskell.orgmailto:haskell-platform@projects.haskell.org; haskell- | infrastructure@community.galois.commailto:infrastructure@community.galois.com; Haskell Libraries; ghc- | devs@haskell.orgmailto:devs@haskell.org | Subject: Cabal and simultaneous installations of the same package | | Dear Cabal developers | | You'll probably have seen the thread about the Haskell Platform. | | Among other things, this point arose: | | | Another thing we should fix is the (now false) impression that HP | | gets in the way of installing other packages and versions due to | cabal hell. | | People mean different things by "cabal hell", but the inability to | simultaneously install multiple versions of the same package, | compiled against different dependencies is certainly one of them, | and I think it is the one that Yitzchak is referring to here. | | But some time now GHC has allowed multiple versions of the same package | (compiled against different dependencies) to be installed | simultaneously. So all we need to do is to fix Cabal to allow it too, | and thereby kill of a huge class of cabal-hell problems at one blow. | | But time has passed and it hasn't happened. Is this because I'm | misunderstanding? Or because it is harder than I think? Or because | there are much bigger problems? Or because there is insufficient | effort available? Or what? | | Unless I'm way off beam, this "multiple installations of the same | package" thing has been a huge pain forever, and the solution is within | our grasp. What's stopping us grasping it? | | Simon | | _______________________________________________ | ghc-devs mailing list | ghc-devs@haskell.orgmailto:ghc-devs@haskell.org | http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Mikhail Glushenkov

5:19 p.m.

On 7 May 2015 at 22:34, Simon Peyton Jones wrote:

...

Dear Cabal developers

I guess everyone is busy, but I feel a bit stuck on knowing how to make progress on this thread.

Vishal Agrawal's GSoC proposal has been accepted. I guess we'll have to wait and see what comes out of it now.

Gershom B

5:28 p.m.

For the info of all, here is the proposal: https://gist.github.com/fugyk/37510958b52589737274 The mentors are Ryan Trinkle and Thomas Tuegel. Hopefully as the project proceeds, Vishal will be checking in with us, asking questions and seeking clarification where necessary, etc. --Gershom On Thu, May 7, 2015 at 5:19 PM, Mikhail Glushenkov < the.dead.shall.rise@gmail.com> wrote:

...

On 7 May 2015 at 22:34, Simon Peyton Jones wrote:

...
Dear Cabal developers

I guess everyone is busy, but I feel a bit stuck on knowing how to make progress on this thread.

Vishal Agrawal's GSoC proposal has been accepted. I guess we'll have to wait and see what comes out of it now. _______________________________________________ cabal-devel mailing list cabal-devel@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/cabal-devel

Simon Peyton Jones

11 May 11 May

6:21 a.m.

I see this as mission-critical to relieving Cabal hell. (Am I alone?) So it’s great that Vishal is going to work on it. Go Vishal! Can I take it, then, that the Cabal developers will be sufficiently involved that, assuming there are no show-stopping problems, they’ll accept the patch? It is tantalising to me that something so critical has been so long delayed. It’d be fantastic if it was done this summer. Simon From: Gershom B [mailto:gershomb@gmail.com] Sent: 07 May 2015 22:29 To: Mikhail Glushenkov Cc: Simon Peyton Jones; haskell-infrastructure@community.galois.com; cabal-devel@haskell.org; Haskell Libraries; ghc-devs@haskell.org Subject: Re: Cabal and simultaneous installations of the same package For the info of all, here is the proposal: https://gist.github.com/fugyk/37510958b52589737274 The mentors are Ryan Trinkle and Thomas Tuegel. Hopefully as the project proceeds, Vishal will be checking in with us, asking questions and seeking clarification where necessary, etc. --Gershom On Thu, May 7, 2015 at 5:19 PM, Mikhail Glushenkov mailto:the.dead.shall.rise@gmail.com> wrote: On 7 May 2015 at 22:34, Simon Peyton Jones mailto:simonpj@microsoft.com> wrote:

...

Dear Cabal developers

I guess everyone is busy, but I feel a bit stuck on knowing how to make progress on this thread.

Vishal Agrawal's GSoC proposal has been accepted. I guess we'll have to wait and see what comes out of it now. _______________________________________________ cabal-devel mailing list cabal-devel@haskell.orgmailto:cabal-devel@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/cabal-devel

Daniel Trstenjak

7:04 a.m.

Hi Simon,

...

It is tantalising to me that something so critical has been so long delayed. It’d be fantastic if it was done this summer.

I'm just wondering what kind of negative side effects it might have. Sure, it will - most likely - make installing of cabal packages a lot easier, especially for all non expert Haskell users. But if multiple versions of the same library can be linked into one binary, then the binary sizes will increase and therefore the memory usage might increase and even performance might decrease a bit if the number of cache misses increase. It's certainly annoying to keep up with library updates, but in a way it also pushes everyone forward to update their code bases, so if everything just "works" - respectively is easy buildable - then this force might be quite reduced. I'm just thinking e.g. of a case where you want to use at least a certain version of a library, because you know of major performance/bug fixes in this version, but now a considerable part of your code base might use an older version of this library, because the dependencies of your project haven't been updated. Greetings, Daniel

Edward Z. Yang

5 Jun 5 Jun

5:31 p.m.

...

But if multiple versions of the same library can be linked into one binary, then the binary sizes will increase and therefore the memory usage might increase and even performance might decrease a bit if the number of cache misses increase.

This is not in the current proposal. GHC knows how to deal with this, but extending Cabal to do this is out of scope for the GSOC. Edward

Herbert Valerio Riedel

29 Aug 29 Aug

2:56 a.m.

New subject: Please help beta test "no-reinstall Cabal" (was: Cabal and simultaneous installations of the same package)

Good news, everyone! ...you may be interested to know this has finally come to fruition (just in time for HIW): http://blog.ezyang.com/2015/08/help-us-beta-test-no-reinstall-cabal/ Cheers, hvr On 2015-03-23 at 09:45:48 +0100, Simon Peyton Jones wrote:

...

Dear Cabal developers

You'll probably have seen the thread about the Haskell Platform.

Among other things, this point arose:

| Another thing we should fix is the (now false) impression that HP gets in | the way of installing other packages and versions due to cabal hell.

People mean different things by "cabal hell", but the inability to simultaneously install multiple versions of the same package, compiled against different dependencies is certainly one of them, and I think it is the one that Yitzchak is referring to here.

But some time now GHC has allowed multiple versions of the same package (compiled against different dependencies) to be installed simultaneously. So all we need to do is to fix Cabal to allow it too, and thereby kill of a huge class of cabal-hell problems at one blow.

But time has passed and it hasn't happened. Is this because I'm misunderstanding? Or because it is harder than I think? Or because there are much bigger problems? Or because there is insufficient effort available? Or what?

Unless I'm way off beam, this "multiple installations of the same package" thing has been a huge pain forever, and the solution is within our grasp. What's stopping us grasping it?

3552

Age (days ago)

3711

Last active (days ago)

List overview

Download

21 comments

11 participants

participants (11)

Daniel Trstenjak
Duncan Coutts
Edward Z. Yang
Gershom B
Herbert Valerio Riedel
Herbert Valerio Riedel
Johan Tibell
Michael Snoyman
Mikhail Glushenkov
Miëtek Bak
Simon Peyton Jones