On Wed, Mar 25, 2015 at 10:24 AM, Michael Snoyman wrote:

On Wed, Mar 25, 2015 at 4:17 PM Anthony Cowley wrote:

...

The suggestion to use "cabal install --dependencies-only ..." instead of "cabal freeze" in that issue is really nicely presented. "cabal freeze" is close to the right thing, but it's just not as fully featured as "cabal install" (e.g. taking flags).

As for Stackage, I think it would be helpful to cache the full build plans computed for each package in Stackage. This is most of the work my Nix tooling currently does, so it would be a big time saver.

By "full build plans," do you mean the dist/setup-config file, or something else? That file would be problematic since it's Cabal-library-version specific IIRC. If you're looking for the full listing of deep dependencies and versions, we can extract that from the .yaml file using the technique I mentioned earlier.

Michael

Yes, I meant the full listing of deep dependencies. Anthony

On Mar 25, 2015, at 10:51 AM, Michael Snoyman wrote:

...

On Wed, Mar 25, 2015 at 4:30 PM Anthony Cowley wrote: On Wed, Mar 25, 2015 at 10:24 AM, Michael Snoyman wrote:

...

On Wed, Mar 25, 2015 at 4:17 PM Anthony Cowley wrote:

...

The suggestion to use "cabal install --dependencies-only ..." instead of "cabal freeze" in that issue is really nicely presented. "cabal freeze" is close to the right thing, but it's just not as fully featured as "cabal install" (e.g. taking flags).

As for Stackage, I think it would be helpful to cache the full build plans computed for each package in Stackage. This is most of the work my Nix tooling currently does, so it would be a big time saver.

By "full build plans," do you mean the dist/setup-config file, or something else? That file would be problematic since it's Cabal-library-version specific IIRC. If you're looking for the full listing of deep dependencies and versions, we can extract that from the .yaml file using the technique I mentioned earlier.

Michael

Yes, I meant the full listing of deep dependencies.

I've put together a Gist with an executable that does what I described:

https://gist.github.com/snoyberg/5b244331533fcb614523

You give it three arguments on the command line:

* LTS version, e.g. 1.14 * Name of package being checked * true == only include dependencies of the library and executable, anything else == include test and benchmark dependencies as well

If that's useful, I can package that up and put it on Hackage.

Michael

This is very helpfulness, thanks! There is a bootstrapping issue, though, which is, I imagine, why both Miëtek and I have been writing much more bash than we'd like. But perhaps this becomes part of a cabal-install-like bootstrap.sh script to get things going. Anthony

On Mar 25, 2015, at 11:03 AM, Michael Snoyman wrote:

...

On Wed, Mar 25, 2015 at 4:58 PM Anthony Cowley wrote:

...

On Mar 25, 2015, at 10:51 AM, Michael Snoyman wrote:

...

On Wed, Mar 25, 2015 at 4:30 PM Anthony Cowley wrote: On Wed, Mar 25, 2015 at 10:24 AM, Michael Snoyman wrote:

...

On Wed, Mar 25, 2015 at 4:17 PM Anthony Cowley wrote:

...

The suggestion to use "cabal install --dependencies-only ..." instead of "cabal freeze" in that issue is really nicely presented. "cabal freeze" is close to the right thing, but it's just not as fully featured as "cabal install" (e.g. taking flags).

As for Stackage, I think it would be helpful to cache the full build plans computed for each package in Stackage. This is most of the work my Nix tooling currently does, so it would be a big time saver.

By "full build plans," do you mean the dist/setup-config file, or something else? That file would be problematic since it's Cabal-library-version specific IIRC. If you're looking for the full listing of deep dependencies and versions, we can extract that from the .yaml file using the technique I mentioned earlier.

Michael

Yes, I meant the full listing of deep dependencies.

I've put together a Gist with an executable that does what I described:

https://gist.github.com/snoyberg/5b244331533fcb614523

You give it three arguments on the command line:

* LTS version, e.g. 1.14 * Name of package being checked * true == only include dependencies of the library and executable, anything else == include test and benchmark dependencies as well

If that's useful, I can package that up and put it on Hackage.

Michael

This is very helpfulness, thanks! There is a bootstrapping issue, though, which is, I imagine, why both Miëtek and I have been writing much more bash than we'd like. But perhaps this becomes part of a cabal-install-like bootstrap.sh script to get things going.

Anthony

Oh, that's actually a great idea. What if we had a program that:

1. takes a set of packages that needs to be installed, and an LTS version 2. computes the dependency tree 3. writes out a shell script (possibly batch program?) to wget, tar xf, and runghc Setup.hs in the correct order to get all of those packages installed

As you can tell from the program I just threw together, stackage-types supports this kind of thing pretty trivially.

Michael

Yes, that's what the Nix tooling does. The extra bits are building up package databases suitable for each build without copying files all over the place. That has the extra benefit of being able to reuse builds when the recipe is unchanged. It is definitely not hard, but getting the build plans from stackage in a portable way would be valuable. Anthony

On Mar 26, 2015, at 3:21 AM, Michael Snoyman wrote:

...

On Wed, Mar 25, 2015 at 7:35 PM Anthony Cowley wrote:

...

On Mar 25, 2015, at 11:03 AM, Michael Snoyman wrote:

...

On Wed, Mar 25, 2015 at 4:58 PM Anthony Cowley wrote:

...

On Mar 25, 2015, at 10:51 AM, Michael Snoyman wrote:

...

On Wed, Mar 25, 2015 at 4:30 PM Anthony Cowley wrote: On Wed, Mar 25, 2015 at 10:24 AM, Michael Snoyman wrote: > > > On Wed, Mar 25, 2015 at 4:17 PM Anthony Cowley > wrote: >> >> The suggestion to use "cabal install --dependencies-only ..." instead >> of "cabal freeze" in that issue is really nicely presented. "cabal >> freeze" is close to the right thing, but it's just not as fully >> featured as "cabal install" (e.g. taking flags). >> >> As for Stackage, I think it would be helpful to cache the full build >> plans computed for each package in Stackage. This is most of the work >> my Nix tooling currently does, so it would be a big time saver. >> >> > > By "full build plans," do you mean the dist/setup-config file, or something > else? That file would be problematic since it's Cabal-library-version > specific IIRC. If you're looking for the full listing of deep dependencies > and versions, we can extract that from the .yaml file using the technique I > mentioned earlier. > > Michael

Yes, I meant the full listing of deep dependencies.

I've put together a Gist with an executable that does what I described:

https://gist.github.com/snoyberg/5b244331533fcb614523

You give it three arguments on the command line:

* LTS version, e.g. 1.14 * Name of package being checked * true == only include dependencies of the library and executable, anything else == include test and benchmark dependencies as well

If that's useful, I can package that up and put it on Hackage.

Michael

This is very helpfulness, thanks! There is a bootstrapping issue, though, which is, I imagine, why both Miëtek and I have been writing much more bash than we'd like. But perhaps this becomes part of a cabal-install-like bootstrap.sh script to get things going.

Anthony

Oh, that's actually a great idea. What if we had a program that:

1. takes a set of packages that needs to be installed, and an LTS version 2. computes the dependency tree 3. writes out a shell script (possibly batch program?) to wget, tar xf, and runghc Setup.hs in the correct order to get all of those packages installed

As you can tell from the program I just threw together, stackage-types supports this kind of thing pretty trivially.

Michael

Yes, that's what the Nix tooling does. The extra bits are building up package databases suitable for each build without copying files all over the place. That has the extra benefit of being able to reuse builds when the recipe is unchanged. It is definitely not hard, but getting the build plans from stackage in a portable way would be valuable.

Anthony

OK, I've updated the Gist so that it produces a shell script that will install all necessary packages. It also has a nicer optparse-applicative interface now.

https://gist.github.com/snoyberg/5b244331533fcb614523

One thing I didn't do here is add support for custom GHC package databases. I assume you'll need that, but wasn't sure exactly how you're doing that in Nix.

If this is useful for you, I'll start a stackage-bootstrap repo, clean up this code a bit, and we can continue improving it from there.

Michael

Just having a URL from which to GET the build plan would be most useful. As you say, the actual package building happens after DB creation, and that requires knowing what to put in the DB. Anthony

Yes, but a line-based format along the lines of: foo 1.2.3 bar-baz 0.1.00 Would be easier to parse with the usual shell tools. Anthony

On Mar 26, 2015, at 9:41 AM, Michael Snoyman wrote:

Is the idea that you'd be able to make a query such as:

GET https://www.stackage.org/lts/1.14/build-plan?package=foo&package=bar&package=baz

And get a result such as:

[ {"name":"foo", "version":"1.2.3"} , ... ]

?

...

On Thu, Mar 26, 2015 at 3:00 PM Anthony Cowley wrote:

...

On Mar 26, 2015, at 3:21 AM, Michael Snoyman wrote:

...

On Wed, Mar 25, 2015 at 7:35 PM Anthony Cowley wrote:

...

On Mar 25, 2015, at 11:03 AM, Michael Snoyman wrote:

...

On Wed, Mar 25, 2015 at 4:58 PM Anthony Cowley wrote:

> On Mar 25, 2015, at 10:51 AM, Michael Snoyman wrote: > > > >> On Wed, Mar 25, 2015 at 4:30 PM Anthony Cowley wrote: >> On Wed, Mar 25, 2015 at 10:24 AM, Michael Snoyman wrote: >> > >> > >> > On Wed, Mar 25, 2015 at 4:17 PM Anthony Cowley >> > wrote: >> >> >> >> The suggestion to use "cabal install --dependencies-only ..." instead >> >> of "cabal freeze" in that issue is really nicely presented. "cabal >> >> freeze" is close to the right thing, but it's just not as fully >> >> featured as "cabal install" (e.g. taking flags). >> >> >> >> As for Stackage, I think it would be helpful to cache the full build >> >> plans computed for each package in Stackage. This is most of the work >> >> my Nix tooling currently does, so it would be a big time saver. >> >> >> >> >> > >> > By "full build plans," do you mean the dist/setup-config file, or something >> > else? That file would be problematic since it's Cabal-library-version >> > specific IIRC. If you're looking for the full listing of deep dependencies >> > and versions, we can extract that from the .yaml file using the technique I >> > mentioned earlier. >> > >> > Michael >> >> Yes, I meant the full listing of deep dependencies. > > I've put together a Gist with an executable that does what I described: > > https://gist.github.com/snoyberg/5b244331533fcb614523 > > You give it three arguments on the command line: > > * LTS version, e.g. 1.14 > * Name of package being checked > * true == only include dependencies of the library and executable, anything else == include test and benchmark dependencies as well > > If that's useful, I can package that up and put it on Hackage. > > Michael

This is very helpfulness, thanks! There is a bootstrapping issue, though, which is, I imagine, why both Miëtek and I have been writing much more bash than we'd like. But perhaps this becomes part of a cabal-install-like bootstrap.sh script to get things going.

Anthony

Oh, that's actually a great idea. What if we had a program that:

1. takes a set of packages that needs to be installed, and an LTS version 2. computes the dependency tree 3. writes out a shell script (possibly batch program?) to wget, tar xf, and runghc Setup.hs in the correct order to get all of those packages installed

As you can tell from the program I just threw together, stackage-types supports this kind of thing pretty trivially.

Michael

Yes, that's what the Nix tooling does. The extra bits are building up package databases suitable for each build without copying files all over the place. That has the extra benefit of being able to reuse builds when the recipe is unchanged. It is definitely not hard, but getting the build plans from stackage in a portable way would be valuable.

Anthony

OK, I've updated the Gist so that it produces a shell script that will install all necessary packages. It also has a nicer optparse-applicative interface now.

https://gist.github.com/snoyberg/5b244331533fcb614523

One thing I didn't do here is add support for custom GHC package databases. I assume you'll need that, but wasn't sure exactly how you're doing that in Nix.

If this is useful for you, I'll start a stackage-bootstrap repo, clean up this code a bit, and we can continue improving it from there.

Michael

Just having a URL from which to GET the build plan would be most useful. As you say, the actual package building happens after DB creation, and that requires knowing what to put in the DB.

Anthony

This is a touch subtle. base can be left out, but other packages that ship with ghc and are usually found in the global package db should be included as they can be upgraded. This has knock on effects where a new version of the unix package means we have to rebuild the directory package with that updated dependency, for example. This is something I already handle, so I'd be happy to have everything included in the provided build plan. I'd need to think more about whether or not some packages can safely be totally left out, but I'm on the road at the moment and not terribly capable of thought. Anthony

On Mar 26, 2015, at 10:50 AM, Michael Snoyman wrote:

And do you want the information on the core packages (e.g., base, template-haskell, containers) as well, or should they be left out?

...

On Thu, Mar 26, 2015 at 4:48 PM Anthony Cowley wrote: Yes, but a line-based format along the lines of:

foo 1.2.3 bar-baz 0.1.00

Would be easier to parse with the usual shell tools.

Anthony

...

On Mar 26, 2015, at 9:41 AM, Michael Snoyman wrote:

Is the idea that you'd be able to make a query such as:

GET https://www.stackage.org/lts/1.14/build-plan?package=foo&package=bar&package=baz

And get a result such as:

[ {"name":"foo", "version":"1.2.3"} , ... ]

?

...

On Thu, Mar 26, 2015 at 3:00 PM Anthony Cowley wrote:

...

On Mar 26, 2015, at 3:21 AM, Michael Snoyman wrote:

...

On Wed, Mar 25, 2015 at 7:35 PM Anthony Cowley wrote:

> On Mar 25, 2015, at 11:03 AM, Michael Snoyman wrote: > > > >> On Wed, Mar 25, 2015 at 4:58 PM Anthony Cowley wrote: >> >> >>> On Mar 25, 2015, at 10:51 AM, Michael Snoyman wrote: >>> >>> >>> >>>> On Wed, Mar 25, 2015 at 4:30 PM Anthony Cowley wrote: >>>> On Wed, Mar 25, 2015 at 10:24 AM, Michael Snoyman wrote: >>>> > >>>> > >>>> > On Wed, Mar 25, 2015 at 4:17 PM Anthony Cowley >>>> > wrote: >>>> >> >>>> >> The suggestion to use "cabal install --dependencies-only ..." instead >>>> >> of "cabal freeze" in that issue is really nicely presented. "cabal >>>> >> freeze" is close to the right thing, but it's just not as fully >>>> >> featured as "cabal install" (e.g. taking flags). >>>> >> >>>> >> As for Stackage, I think it would be helpful to cache the full build >>>> >> plans computed for each package in Stackage. This is most of the work >>>> >> my Nix tooling currently does, so it would be a big time saver. >>>> >> >>>> >> >>>> > >>>> > By "full build plans," do you mean the dist/setup-config file, or something >>>> > else? That file would be problematic since it's Cabal-library-version >>>> > specific IIRC. If you're looking for the full listing of deep dependencies >>>> > and versions, we can extract that from the .yaml file using the technique I >>>> > mentioned earlier. >>>> > >>>> > Michael >>>> >>>> Yes, I meant the full listing of deep dependencies. >>> >>> I've put together a Gist with an executable that does what I described: >>> >>> https://gist.github.com/snoyberg/5b244331533fcb614523 >>> >>> You give it three arguments on the command line: >>> >>> * LTS version, e.g. 1.14 >>> * Name of package being checked >>> * true == only include dependencies of the library and executable, anything else == include test and benchmark dependencies as well >>> >>> If that's useful, I can package that up and put it on Hackage. >>> >>> Michael >> >> This is very helpfulness, thanks! There is a bootstrapping issue, though, which is, I imagine, why both Miëtek and I have been writing much more bash than we'd like. But perhaps this becomes part of a cabal-install-like bootstrap.sh script to get things going. >> >> Anthony > > Oh, that's actually a great idea. What if we had a program that: > > 1. takes a set of packages that needs to be installed, and an LTS version > 2. computes the dependency tree > 3. writes out a shell script (possibly batch program?) to wget, tar xf, and runghc Setup.hs in the correct order to get all of those packages installed > > As you can tell from the program I just threw together, stackage-types supports this kind of thing pretty trivially. > > Michael

Yes, that's what the Nix tooling does. The extra bits are building up package databases suitable for each build without copying files all over the place. That has the extra benefit of being able to reuse builds when the recipe is unchanged. It is definitely not hard, but getting the build plans from stackage in a portable way would be valuable.

Anthony

OK, I've updated the Gist so that it produces a shell script that will install all necessary packages. It also has a nicer optparse-applicative interface now.

https://gist.github.com/snoyberg/5b244331533fcb614523

One thing I didn't do here is add support for custom GHC package databases. I assume you'll need that, but wasn't sure exactly how you're doing that in Nix.

If this is useful for you, I'll start a stackage-bootstrap repo, clean up this code a bit, and we can continue improving it from there.

Michael

Just having a URL from which to GET the build plan would be most useful. As you say, the actual package building happens after DB creation, and that requires knowing what to put in the DB.

Anthony

OK, should be working now: http://www.stackage.org/lts/build-plan?package=warp On Thu, Mar 26, 2015 at 5:03 PM Michael Snoyman wrote:

It's trivial for me to include them all, and equally trivial for me to call out which ones are shipped with GHC somehow (such as an extra flag on that line). I'll implement this currently to dump everything out without any extra flag, and we can tweak it in the future.

On Thu, Mar 26, 2015 at 5:01 PM Anthony Cowley wrote:

...

This is a touch subtle. base can be left out, but other packages that ship with ghc and are usually found in the global package db should be included as they can be upgraded. This has knock on effects where a new version of the unix package means we have to rebuild the directory package with that updated dependency, for example. This is something I already handle, so I'd be happy to have everything included in the provided build plan. I'd need to think more about whether or not some packages can safely be totally left out, but I'm on the road at the moment and not terribly capable of thought.

Anthony

On Mar 26, 2015, at 10:50 AM, Michael Snoyman wrote:

And do you want the information on the core packages (e.g., base, template-haskell, containers) as well, or should they be left out?

On Thu, Mar 26, 2015 at 4:48 PM Anthony Cowley wrote:

...

Yes, but a line-based format along the lines of:

foo 1.2.3 bar-baz 0.1.00

Would be easier to parse with the usual shell tools.

Anthony

On Mar 26, 2015, at 9:41 AM, Michael Snoyman wrote:

Is the idea that you'd be able to make a query such as:

GET https://www.stackage.org/lts/1.14/build-plan?package=foo& package=bar&package=baz

And get a result such as:

[ {"name":"foo", "version":"1.2.3"} , ... ]

?

On Thu, Mar 26, 2015 at 3:00 PM Anthony Cowley wrote:

...

On Mar 26, 2015, at 3:21 AM, Michael Snoyman wrote:

On Wed, Mar 25, 2015 at 7:35 PM Anthony Cowley wrote:

...

On Mar 25, 2015, at 11:03 AM, Michael Snoyman wrote:

On Wed, Mar 25, 2015 at 4:58 PM Anthony Cowley wrote:

...

On Mar 25, 2015, at 10:51 AM, Michael Snoyman wrote:

On Wed, Mar 25, 2015 at 4:30 PM Anthony Cowley < acowley@seas.upenn.edu> wrote:

> On Wed, Mar 25, 2015 at 10:24 AM, Michael Snoyman < > michael@snoyman.com> wrote: > > > > > > On Wed, Mar 25, 2015 at 4:17 PM Anthony Cowley < > acowley@seas.upenn.edu> > > wrote: > >> > >> The suggestion to use "cabal install --dependencies-only ..." > instead > >> of "cabal freeze" in that issue is really nicely presented. "cabal > >> freeze" is close to the right thing, but it's just not as fully > >> featured as "cabal install" (e.g. taking flags). > >> > >> As for Stackage, I think it would be helpful to cache the full > build > >> plans computed for each package in Stackage. This is most of the > work > >> my Nix tooling currently does, so it would be a big time saver. > >> > >> > > > > By "full build plans," do you mean the dist/setup-config file, or > something > > else? That file would be problematic since it's > Cabal-library-version > > specific IIRC. If you're looking for the full listing of deep > dependencies > > and versions, we can extract that from the .yaml file using the > technique I > > mentioned earlier. > > > > Michael > > Yes, I meant the full listing of deep dependencies. > > > I've put together a Gist with an executable that does what I described:

https://gist.github.com/snoyberg/5b244331533fcb614523

You give it three arguments on the command line:

* LTS version, e.g. 1.14 * Name of package being checked * true == only include dependencies of the library and executable, anything else == include test and benchmark dependencies as well

If that's useful, I can package that up and put it on Hackage.

Michael

This is very helpfulness, thanks! There is a bootstrapping issue, though, which is, I imagine, why both Miëtek and I have been writing much more bash than we'd like. But perhaps this becomes part of a cabal-install-like bootstrap.sh script to get things going.

Anthony

Oh, that's actually a great idea. What if we had a program that:

1. takes a set of packages that needs to be installed, and an LTS version 2. computes the dependency tree 3. writes out a shell script (possibly batch program?) to wget, tar xf, and runghc Setup.hs in the correct order to get all of those packages installed

As you can tell from the program I just threw together, stackage-types supports this kind of thing pretty trivially.

Michael

Yes, that's what the Nix tooling does. The extra bits are building up package databases suitable for each build without copying files all over the place. That has the extra benefit of being able to reuse builds when the recipe is unchanged. It is definitely not hard, but getting the build plans from stackage in a portable way would be valuable.

Anthony

OK, I've updated the Gist so that it produces a shell script that will install all necessary packages. It also has a nicer optparse-applicative interface now.

https://gist.github.com/snoyberg/5b244331533fcb614523

One thing I didn't do here is add support for custom GHC package databases. I assume you'll need that, but wasn't sure exactly how you're doing that in Nix.

If this is useful for you, I'll start a stackage-bootstrap repo, clean up this code a bit, and we can continue improving it from there.

Michael

Just having a URL from which to GET the build plan would be most useful. As you say, the actual package building happens after DB creation, and that requires knowing what to put in the DB.

Anthony