Re: Agreeing on a UI for sandboxes

Hi all, I've should have CCed cabal-devel on this discussion from the start. My apologies. I will try to remember to have all these discussion in the open in the future. We're discussing how the sandbox mechanism that cabal got during this year's GSoC should be exposed to users. I've included the discussion so far below: --8<----------------------- Date: Fri, 24 Aug 2012 15:40:14 -0700 Subject: Agreeing on a UI for sandboxes From: Johan Tibell To: Duncan Coutts , =?ISO-8859-1?Q?Andres_L=F6h?= , Mikhail Glushenkov Hi all, I've now merged Mikhail's last patches and the remaining work is to figure out how to expose the sandbox mechanism to users in a way that easy to understand and convenient to use. Mikhail has added some hidden commands that you can use to today to work within the sandbox. The goal is to remove these commands in favor of modifying the behavior of the current command set. If we could bring out the different possible designs and issues before ICFP, perhaps we can solve any remaining issues in person there. Mikhail, as a starting point, could you please explain how you'd use these hidden commands (you can probably reuse your description from the pull request). -- Johan --8<----------------------- Date: Sat, 25 Aug 2012 01:01:04 +0200 Subject: Re: Agreeing on a UI for sandboxes From: Mikhail Glushenkov To: Johan Tibell Cc: Duncan Coutts , =?ISO-8859-1?Q?Andres_L=F6h?= Hi all, On Sat, Aug 25, 2012 at 12:40 AM, Johan Tibell wrote:

Mikhail, as a starting point, could you please explain how you'd use these hidden commands (you can probably reuse your description from the pull request).

There are four new hidden commands: * cabal sandbox-configure Same as 'cabal configure', but also initialises a new sandbox to be used for building. * cabal sandbox-add-source Makes a source package available for installation inside the sandbox. * cabal sandbox-build Builds a package inside the sandbox. * cabal sandbox-install Installs a package into the sandbox. I tried to keep the UI close to the current one, so dependency installation & configuration is still done manually. If a dependency added with sandbox-add-source is updated, it must be manually reinstalled with cabal sandbox-install LibName. Here's an example session: $ git clone git://github.com/haskell/cabal.git $ cd cabal/cabal-install $ cabal sandbox-configure -w /path/to/ghc-7.4.1 $ cabal sandbox-add-source ../Cabal $ cabal sandbox-install --only-dependencies # Since the previous configure only initialised the sandbox $ cabal sandbox-configure -w /path/to/ghc-7.4.1 $ cabal sandbox-build One major annoyance with the current UI is that you often have to run cabal sandbox-configure twice (first to initialise the sandbox, and after that to actually configure the package once the dependencies have been installed). I propose that we make sandbox-configure run 'sandbox-install --only-dependencies' automatically after creating a sandbox. --8<----------------------- Date: Mon, 27 Aug 2012 20:28:29 +0200 Message-ID: Subject: Re: Agreeing on a UI for sandboxes From: =?ISO-8859-1?Q?Andres_L=F6h?= To: Mikhail Glushenkov Cc: Johan Tibell , Duncan Coutts Hi.

I tried to keep the UI close to the current one, so dependency installation & configuration is still done manually. If a dependency added with sandbox-add-source is updated, it must be manually reinstalled with cabal sandbox-install LibName.

You mean a sandbox-install command to something depending on LibName won't pick it up?

Here's an example session:

$ git clone git://github.com/haskell/cabal.git $ cd cabal/cabal-install $ cabal sandbox-configure -w /path/to/ghc-7.4.1

So I can only create a sandbox in a Cabal package directory? And "cabal sandbox-add-source ." is implicit?

$ cabal sandbox-add-source ../Cabal $ cabal sandbox-install --only-dependencies

Why can't I just say "cabal sandbox-install" at this point?

# Since the previous configure only initialised the sandbox $ cabal sandbox-configure -w /path/to/ghc-7.4.1 $ cabal sandbox-build

One major annoyance with the current UI is that you often have to run cabal sandbox-configure twice (first to initialise the sandbox, and after that to actually configure the package once the dependencies have been installed). I propose that we make sandbox-configure run 'sandbox-install --only-dependencies' automatically after creating a sandbox.

I'm not particularly fond of having non-install commands download stuff from the internet. Cheers, Andres --8<----------------------- Date: Tue, 28 Aug 2012 03:51:59 +0200 Subject: Re: Agreeing on a UI for sandboxes From: Mikhail Glushenkov To: =?ISO-8859-1?Q?Andres_L=F6h?= Cc: Johan Tibell , Duncan Coutts Hi, On Mon, Aug 27, 2012 at 8:28 PM, Andres L=F6h = wrote:

Hi.

...

I tried to keep the UI close to the current one, so dependency installation & configuration is still done manually. If a dependency added with sandbox-add-source is updated, it must be manually reinstalled with cabal sandbox-install LibName.

You mean a sandbox-install command to something depending on LibName won't pick it up?

Not if the same version is already installed. We can change it so all dependencies added with add-source are always reinstalled when you run sandbox-install. I'm not quite satisfied with both the current and the proposed solution, ideally it'd possible to detect when some .hs files have changed and the package needs to be reinstalled.

...

Here's an example session:

$ git clone git://github.com/haskell/cabal.git $ cd cabal/cabal-install $ cabal sandbox-configure -w /path/to/ghc-7.4.1

So I can only create a sandbox in a Cabal package directory?

There is a --sandbox option that lets you specify the location (so shared sandboxes are allowed).

And "cabal sandbox-add-source ." is implicit?

This is not currently being done, but is probably worth adding (to support shared sandboxes better).

...

$ cabal sandbox-add-source ../Cabal $ cabal sandbox-install --only-dependencies

Why can't I just say "cabal sandbox-install" at this point?

You can use sandbox-install once the sandbox has been created, it's just that I opted to use sandbox-build in the example.

I'm not particularly fond of having non-install commands download stuff from the internet.

Good point, I'm also a bit uneasy about this. On the other hand, running 'sandbox-install --only-dependencies' directly after creating a sandbox is a common use case. I also don't like requiring the user to run 'sandbox-configure' twice. Maybe the sandbox should be initialised by install instead of configure? --8<----------------------- Date: Tue, 28 Aug 2012 08:28:47 +0200 Subject: Re: Agreeing on a UI for sandboxes From: =?ISO-8859-1?Q?Andres_L=F6h?= To: Mikhail Glushenkov Cc: Johan Tibell , Duncan Coutts Hi.

...

You mean a sandbox-install command to something depending on LibName won't pick it up?

Not if the same version is already installed.

Ah, ok. But if the version changes, it will. That makes sense. The other point would be fixed if the other GSoC project gets closer to maturity.

We can change it so all dependencies added with add-source are always reinstalled when you run sandbox-install.

I wouldn't change it yet. If at all, it should be a flag per add-source. Not everything you add-source are libraries you necessarily actively develop at the time. And I guess you can add-source not only local packages, right?

I'm not quite satisfied with both the current and the proposed solution, ideally it'd possible to detect when some .hs files have changed and the package needs to be reinstalled.

Yes, indeed.

...

So I can only create a sandbox in a Cabal package directory?

There is a --sandbox option that lets you specify the location (so shared sandboxes are allowed).

Cool.

...

And "cabal sandbox-add-source ." is implicit?

This is not currently being done, but is probably worth adding (to support shared sandboxes better).

...

...

$ cabal sandbox-add-source ../Cabal $ cabal sandbox-install --only-dependencies

Why can't I just say "cabal sandbox-install" at this point?

You can use sandbox-install once the sandbox has been created, it's just that I opted to use sandbox-build in the example.

Yes, but that's what makes you end up having to run sanbox-configure twice.

...

I'm not particularly fond of having non-install commands download stuff from the internet.

Good point, I'm also a bit uneasy about this. On the other hand, running 'sandbox-install --only-dependencies' directly after creating a sandbox is a common use case. I also don't like requiring the user to run 'sandbox-configure' twice. Maybe the sandbox should be initialised by install instead of configure?

That seems to be more in line with how it works without sandbox-* ? Cheers, Andres --8<----------------------- Date: Tue, 28 Aug 2012 16:55:46 +0200 Subject: Re: Agreeing on a UI for sandboxes From: Mikhail Glushenkov To: =?ISO-8859-1?Q?Andres_L=F6h?= Cc: Johan Tibell , Duncan Coutts Hi, On Tue, Aug 28, 2012 at 8:28 AM, Andres L=F6h = wrote:

Ah, ok. But if the version changes, it will. That makes sense. The other point would be fixed if the other GSoC project gets closer to maturity.

Looking forward to that!

I wouldn't change it yet. If at all, it should be a flag per add-source. Not everything you add-source are libraries you necessarily actively develop at the time.

We discussed this - the initial idea was to have a separate link-source command. Johan thinks that it is redundant and it would be best to avoid adding it. One idea is to make 'install' run 'cabal build' on packages added with add-source (those that we depend on) and then only install those files that have actually changed (so that re-linking is avoided).

And I guess you can add-source not only local packages, right?

The plan is to allow linking to remote repositories. This is not yet implemented.

...

...

I'm not particularly fond of having non-install commands download stuff from the internet.

Good point, I'm also a bit uneasy about this. On the other hand, running 'sandbox-install --only-dependencies' directly after creating a sandbox is a common use case. I also don't like requiring the user to run 'sandbox-configure' twice. Maybe the sandbox should be initialised by install instead of configure?

That seems to be more in line with how it works without sandbox-* ?

Kind of. I think that I'll make sandbox-install initialise a sandbox by default because of this use case: $ mkdir darcs && cd darcs $ cabal sandbox-install darcs But using 'install' for building feels inconsistent. For the normal build you are using 'cabal configure && cabal install --only-dependencies && cabal build'. For the sandboxed build you would be using 'cabal install --sandbox'. Ideally, I'd like the differences between workflows to be minimal. Assuming that the standard commands were made sandbox-aware: $ cabal configure --sandbox $ cabal install --only-dependencies $ cabal build But configure fails when the sandbox is empty, so one has to run it twice. --8<----------------------- Date: Tue, 28 Aug 2012 16:45:55 -0700 Subject: Re: Agreeing on a UI for sandboxes From: Johan Tibell To: Mikhail Glushenkov Cc: =?ISO-8859-1?Q?Andres_L=F6h?= , Duncan Coutts Hi, I haven't had time to collect my thoughts fully yet, so here are some initial thoughts: It's important that users don't use a mix of sandbox and non-sandbox commands by misstake. That would lead to confusing build errors. Ideally the users should request that work is to be done in a sandbox once and then all commands should use the sandbox. This means that there should be no e.g. cabal build/test/bench --sandbox flags. Perhaps a separate command, cabal init-sandbox, would be the best approach. Running that command would create the right package environment for running in a sandbox and the remaining commands would then use that sandbox. Separating init-sandbox from configure prevents the user from removing the sandbox preference when reconfiguring for some other reason (e.g. to add --enable-tests). To get out of the sandbox the user would edit the package environment file. Alternatively we could have a sandbox command that lets you both add and remove sandboxes. cabal sandbox --init cabal sandbox --delete cabal sandbox --init --path=/path/to/sandbox -- Johan --8<----------------------- Date: Tue, 28 Aug 2012 16:56:20 -0700 Subject: Re: Agreeing on a UI for sandboxes From: Johan Tibell To: =?ISO-8859-1?Q?Andres_L=F6h?= Cc: Mikhail Glushenkov , Duncan Coutts On Mon, Aug 27, 2012 at 11:28 PM, Andres L=F6h = wrote:

...

We can change it so all dependencies added with add-source are always reinstalled when you run sandbox-install.

I wouldn't change it yet. If at all, it should be a flag per add-source. Not everything you add-source are libraries you necessarily actively develop at the time. And I guess you can add-source not only local packages, right?

I'm worried about making cabal even more cumbersome to use. Today you need to do the following to work on several repos at once: for every dependency: cd $dir cabal configure cabal build cd $main-dir cabal configure --package-db=3D$dir1/dist/package.conf.inplace --package-db=3D$dir1/dist/package.conf.inplace ... If the use manually has to rebuild dependencies every time the change even when using a sandbox we have essentially no improvement over status quo and or cabal-dev. Cabal today is too much about mechanism and not enough about user intentions. Lets try to move it towards the latter, not away from it. Here's my claim, if rebuilding my project requires something more than "cabal build", we did something wrong in designing the user interface. -- Johan