On Fri, Nov 19, 2010 at 7:01 AM, Duncan Coutts
On Fri, 2010-11-19 at 12:27 +0000, Duncan Coutts wrote:
Matt and I also discussed making the 00-index.tar.gz into a RESTful format by adding proper URLs for package tarballs.
Indeed we could go further and use a single general format for describing or distributing bundles of packages.
Use case: local build trees ---------------------------
A bunch of related packages (e.g. gtk2hs, happstack-* etc) unpacked locally.
/home/me/prgs/myproj/foo/ --top of source tree for foo /home/me/prgs/myproj/foo/foo.cabal /home/me/prgs/myproj/bar/ /home/me/prgs/myproj/bar/bar.cabal
Now we can have an index.tar containing symlinks to .cabal files!
/home/me/prgs/myproj/index.tar: containing foo.cabal -> foo/foo.cabal bar.cabal -> bar/bar.cabal
So these are not copies of the .cabal files, these really are symlinks to the local .cabal files (but inside the tarball). I guess we need some extra index entry to point to the location of the source tree, though it's not a .tar.gz kind.
Now just as we can have symlinks (or really URLs) inside the tarball, we could also have full file contents there too. Next use case...
Use case: distribution bundles ------------------------------
Shipping a bunch of source packages as a single file
some-name.tar: containing foo.cabal foo-1.0.tar.gz bar.cabal bar-1.0.tar.gz
So now instead of symlinks/URLs to separate tarballs, the whole file contents is right there. We have a hackage-like index plus the file tarballs.
We might have to have a different naming convention than simply blah.tar for these indexes, otherwise cabal install might not know how to interpret "cabal install foo.tar" should it interpret foo.tar as an index or as a single package?
Opinions?
It feels like an abuse of tar-files to me - if we want to have a set of meta-data about the location of resources in a package repository, I think it would be better to come up with a file format that has the information we want directly and then serve it up. This hypothetical cabal-repository.description file would be pointed at by a user's .cabal/conf, and the config file would describe either what resources the repo makes available or how to discover what resources it makes available. So for a small repo, this file could contain a listing of package ids and where the tar-ball/package descriptions are. We could even have a special case for local or file-share hosted repositories - the presence of an empty repo description file would imply that the contents of the repo is every tar, tar.gz or directory containing a .cabal file in the top level. A larger repository would point to another file which contains a collection of packages and their meta-data. One of the resources could be "here's where to find a tarball containing the package descriptions of every package I know how to serve" to support the current model of solving dependencies based. In this scenario the 'repo description' files would exactly be a REST description of the contents of Hackage Server. It's the same information as what you'd wanted to put in the index tarball, and we might even want to make it so that the repo config file can live in the tarball and address resources in the tarball it is hosted in (so I can deply a local cabal repo by dropping a tarball into a fileshare). But slipstreaming metadata into soft-links in a tarball feels weird, and since we need client changes to make it work we may as well do it right. Does this sort of approach sound sensible? I don't mind fleshing it out more as a start. Antoine