- cabal-devel - Haskell.org

patch applied (hackage-server): "Make PackageCandidates use TarIndexCache"
by devnull＠community.haskell.org 23 Jan '13

23 Jan '13

Wed Jan 23 12:14:32 GMT 2013 Edsko de Vries <edsko(a)well-typed.com> * Make PackageCandidates use TarIndexCache Ignore-this: f36901d0ab55d1f1625b97ef5b0471c3 M ./Distribution/Server/Features.hs +1 M ./Distribution/Server/Features/PackageCandidates.hs -68 +63

1 0

patch applied (hackage-server): "Introduce TarIndexCache feature"
by devnull＠community.haskell.org 23 Jan '13

23 Jan '13

Wed Jan 23 11:40:35 GMT 2013 Edsko de Vries <edsko(a)well-typed.com> * Introduce TarIndexCache feature Ignore-this: 9450fc9a76131a0fa21329f67d4a6540 This provides generic support for caching tarball indices. The package contents feature now uses this to get access to the tar index. We no longer reconstruct the tar index on every request! Whoohoo :) M ./Distribution/Server/Features.hs +4 R ./Distribution/Server/Features/PackageContents/ M ./Distribution/Server/Features/PackageContents.hs -65 +51 R ./Distribution/Server/Features/PackageContents/State.hs A ./Distribution/Server/Features/TarIndexCache/ A ./Distribution/Server/Features/TarIndexCache.hs A ./Distribution/Server/Features/TarIndexCache/State.hs M ./hackage-server.cabal +3

1 0

patch applied (hackage-server): "Remove dependency of PackageContents on PackageCandidates" and 2 others
by devnull＠community.haskell.org 22 Jan '13

22 Jan '13

Tue Jan 22 14:40:39 GMT 2013 Edsko de Vries <edsko(a)well-typed.com> * Remove dependency of PackageContents on PackageCandidates Ignore-this: f6eb56b31f5e55cd3078dfdd0e4560c3 This means there is a bit of duplication between PackageCandidates and PackageCandidates (serveTarball and friends), but we can factor that out later. Importantly, both PackageContents and PackageCandidates can now maintain their own state (cached TarIndexes) so that we can start to get rid of the stuff in Util/ChangeLog (which reconstructs the TarIndex on every request). M ./Distribution/Server/Features.hs -7 +6 M ./Distribution/Server/Features/PackageCandidates.hs -11 +61 M ./Distribution/Server/Features/PackageContents.hs -32 +11 M ./Distribution/Server/Packages/Render.hs -36 +40 M ./Distribution/Server/Pages/Package.hs -19 +28 Tue Jan 22 15:12:00 GMT 2013 Edsko de Vries <edsko(a)well-typed.com> * Merge Util/Changelog with PackageContents and PackageCandidates Ignore-this: 60d1c81c483ec093a9decadbb266528d This paves the way for making lookupTarball and co stateful (that is, find the cached tarindex rather than reconstructing it). We need to do this separately for PackageContents and PackageCandidates because they will each maintain their own state. This introduces more duplication, but we can separate that out later. M ./Distribution/Server/Features/PackageCandidates.hs -7 +45 M ./Distribution/Server/Features/PackageContents.hs -5 +43 M ./Distribution/Server/Features/RecentPackages.hs -1 +1 R ./Distribution/Server/Util/ChangeLog.hs M ./hackage-server.cabal -1 Tue Jan 22 17:14:36 GMT 2013 Edsko de Vries <edsko(a)well-typed.com> * Add packageTarball and packageChangeLog to PackageContents Ignore-this: 667d26a05a81cbec3e10c75cb2d8c86c At the moment these are simply defined in terms of lookupTarballAndConstructTarIndex and lookupChangeLog, but those functions are now no longer exported (RecentPackages now depends on the PackageContents feature to get a handle on a package changelog). This means that we are now finally in a position that we can actually start caching these tar indices. M ./Distribution/Server/Features.hs -3 +4 A ./Distribution/Server/Features/PackageContents/ M ./Distribution/Server/Features/PackageContents.hs -8 +32 A ./Distribution/Server/Features/PackageContents/State.hs M ./Distribution/Server/Features/RecentPackages.hs -13 +22

1 0

patch applied (hackage-server): "Rename functions to that tarindex creation is more explicit" and 2 others
by devnull＠community.haskell.org 22 Jan '13

22 Jan '13

Tue Jan 22 10:57:26 GMT 2013 Edsko de Vries <edsko(a)well-typed.com> * Rename functions to that tarindex creation is more explicit Ignore-this: 2792e75ff8113a067e4c7a0a5c521298 In particular, readTarIndex is now called constructTarIndexFromFile, and lookupTarball is now given the (intentionally unwieldy) name lookupTarballAndConstructTarIndex. The point is to make it more obvious in the code where we are recreating tar indices (and where we should be caching them, instead). That function lookupTarballAndConstructTarIndex should probably go completely. M ./Distribution/Server/Features/Documentation.hs -1 +1 M ./Distribution/Server/Features/PackageContents.hs -2 +2 M ./Distribution/Server/Util/ChangeLog.hs -6 +8 M ./Distribution/Server/Util/ServeTarball.hs -3 +3 Tue Jan 22 13:33:34 GMT 2013 Edsko de Vries <edsko(a)well-typed.com> * Make links in /api clickable Ignore-this: c6e67378658fd52d512a906bc571010b M ./Distribution/Server/Features/ServerIntrospect.hs -13 +21 Tue Jan 22 13:34:55 GMT 2013 Edsko de Vries <edsko(a)well-typed.com> * Add missing file Ignore-this: b92a11dd3f5e2ea9859f14d1e46d0253 A ./Distribution/Server/Packages/Render.hs

1 0

Sterner warning about breakage with --force-reinstalls?
by Daniel Fischer 14 Jan '13

14 Jan '13

Maybe Warning: The following packages are likely to be broken by the reinstalls: ... Use --force-reinstalls if you want to install anyway. is a bit too inviting for new users, cf. http://stackoverflow.com/questions/14326112/haskell-module-not-found-hidden- package Perhaps it should read "if you really know what you're doing" or some such.

2 1

patch applied (hackage-server): "Rename Packages to RecentPackages and Check to PackageCandidates" and 1 others
by devnull＠community.haskell.org 11 Jan '13

11 Jan '13

Fri Jan 11 14:43:34 GMT 2013 Edsko de Vries <edsko(a)well-typed.com> * Rename Packages to RecentPackages and Check to PackageCandidates Ignore-this: 5b75163b27213b904d4d27616db714d8 ./Distribution/Server/Features/Check -> ./Distribution/Server/Features/PackageCandidates ./Distribution/Server/Features/Check.hs -> ./Distribution/Server/Features/PackageCandidates.hs ./Distribution/Server/Features/Packages.hs -> ./Distribution/Server/Features/RecentPackages.hs M ./Distribution/Server/Features.hs -10 +10 M ./Distribution/Server/Features/Html.hs -9 +9 M ./Distribution/Server/Features/PackageCandidates.hs -19 +19 M ./Distribution/Server/Features/PackageCandidates/State.hs -2 +2 M ./Distribution/Server/Features/PackageCandidates/Types.hs -1 +1 M ./Distribution/Server/Features/PackageContents.hs -4 +4 M ./Distribution/Server/Features/RecentPackages.hs -9 +9 M ./Distribution/Server/Features/Tags.hs -1 +1 M ./Distribution/Server/Pages/Package.hs -1 +1 M ./hackage-server.cabal -4 +4 Fri Jan 11 16:39:28 GMT 2013 Edsko de Vries <edsko(a)well-typed.com> * More renaming. Disentangle some dependencies Ignore-this: b742c261d5ec6c3279cbca98e1a8e50b What we want to do is move the 'lookupTarIndex' to the feature API of the package contents feature, so that we can cache the tarball indexes. However, this will introduce a circular dependency between the package contents feaure and the package candidates feature. More generally, there is quite a bit of overlap between these two features. We should abstract out the common functionality and then disentangle the circular dependencies. This will require a bit more thought though so not doing that right now. M ./Distribution/Server/Features/Html.hs -10 +11 M ./Distribution/Server/Features/PackageCandidates.hs -16 +28 M ./Distribution/Server/Features/PackageContents.hs -3 +6 M ./Distribution/Server/Features/RecentPackages.hs -237 +22 M ./Distribution/Server/Features/Tags.hs -1 +1 M ./Distribution/Server/Pages/Package.hs -1 +1

1 0

patch applied (hackage-server): "Improve the utils for validating and consuming the request body" and 2 others
by devnull＠community.haskell.org 09 Jan '13

09 Jan '13

Wed Jan 9 20:25:36 GMT 2013 Duncan Coutts <duncan(a)well-typed.com> * Improve the utils for validating and consuming the request body Ignore-this: b217bd39e4245315dcf48ef034544f2e Check the mime type better, do transparent gzip decoding and also return the actual content. Also rename the module and equiv response module to be more symetric. ./Distribution/Server/Framework/ResourceTypes.hs -> ./Distribution/Server/Framework/ResponseContentTypes.hs ./Distribution/Server/Framework/Validation.hs -> ./Distribution/Server/Framework/RequestContentTypes.hs M ./Distribution/Server/Features/BuildReports.hs -3 +3 M ./Distribution/Server/Features/Check.hs -1 +1 M ./Distribution/Server/Features/Core.hs -1 +1 M ./Distribution/Server/Features/Distro.hs -4 +5 M ./Distribution/Server/Features/Documentation.hs -4 +3 M ./Distribution/Server/Features/Html.hs -1 +1 M ./Distribution/Server/Features/Mirror.hs -8 +4 M ./Distribution/Server/Features/NameSearch.hs -1 +1 M ./Distribution/Server/Features/Packages.hs -1 +1 M ./Distribution/Server/Features/PreferredVersions.hs -1 +1 M ./Distribution/Server/Features/ServerIntrospect.hs -1 +1 M ./Distribution/Server/Features/Users.hs -3 +2 M ./Distribution/Server/Framework.hs -4 +4 M ./Distribution/Server/Framework/Auth.hs -1 +1 M ./Distribution/Server/Framework/Error.hs +4 M ./Distribution/Server/Framework/RequestContentTypes.hs -32 +98 M ./Distribution/Server/Framework/ResponseContentTypes.hs -6 +6 M ./Distribution/Server/Util/Happstack.hs -2 +3 M ./hackage-server.cabal -2 +3 Wed Jan 9 21:22:06 GMT 2013 Duncan Coutts <duncan(a)well-typed.com> * Rework the documentation upload and download Ignore-this: 662d756bc55eedf2860122eb8394851a It works now, including import. We can import all the docs from the old hackage rather than having to rebuild them all. We do http redirects for all the old links. M ./Data/TarIndex.hs -5 +9 M ./Distribution/Server/Features/Documentation.hs -18 +23 M ./Distribution/Server/Features/Html.hs -1 +1 M ./Distribution/Server/Features/LegacyRedirects.hs -1 +1 M ./Distribution/Server/Util/ServeTarball.hs -16 +18 M ./ImportClient.hs -19 +45 Wed Jan 9 21:25:45 GMT 2013 Duncan Coutts <duncan(a)well-typed.com> * In the hackage download script, also get the html docs Ignore-this: e31cfb27f07da01eae1d5ee902ea523f M ./old-hackage-download.sh -1 +1

1 0

Adding tests for cabal-install
by Mikhail Glushenkov 01 Jan '13

01 Jan '13

Hi all, I'm currently looking into #1159, and I'd really like to add some unit tests for the UserConstraints parser. The problem is that I can't access modules defined under Distribution.Client.* without doing unnecessary recompilation. Would it be fine if I split cabal-install into library and executable parts in the following fashion: Library cabal-install Build-depends: ... ... Executable cabal-install Main-is: Main.hs Build-depends: cabal-install == N.N, ... ... Test-suite unit-tests Build-depends: cabal-install == N.N, ... ... -- () ascii ribbon campaign - against html e-mail /\ www.asciiribbon.org - against proprietary attachments

4 6

hackage-server: index format
by Antoine Latter 20 Dec '12

20 Dec '12

Hi folks, The index tar-ball on Hackage has an odd naming convention. Package descriptions are given paths of the form: ./$pkg/$version/$pkg.cabal including the leading "./". I'm guessing that this is done as a method of distinguishing non-package meta-data. Is this a convention we need to preserve? Thanks, Antoine

6 12

current hackage 2 status
by Duncan Coutts 18 Dec '12

18 Dec '12

Hi folks, You may have noticed the steady stream of hackage 2 patches. The new server is up again at: http://new-hackage.haskell.org/ Key points: * We've got live mirroring going (sync every 30 min). * All the old accounts are imported and the per-package maintainer groups are populated. * Package deprecation info is imported * Distro info is imported * Currently uploading is disabled (pending review of authentication & authorisation code). * Documentation is not imported (Ian demoed this before, just haven't got round to it yet for this instance) * The data backup system is much improved. In general the switch over plan remains that we run the two servers in parallel for long enough to sort out remaining issues (with help from the kind folks who volunteered as admins). There was some discussion before about user accounts and the issue of per-package maintainer groups. So the system I envisage for user accounts is that anyone can make an account, but to be added to the uploaders group will require approval from a volunteer administrator (technically speaking there are not yet any features that need an account but don't involve uploading, but I'm assuming some future social features will need this). I do think the per-package maintainer groups are important for security, and now that we can automatically populate them from the old hackage data, I don't think that's a problem. Still missing is a proper process for new account creation. That is, account creation with email validation and all that. There's a couple of options here. We can build it into the server, or we can make a separate page/script to do that (which would make accounts using the appropriate curl -X PUT commands against the server). -- Duncan Coutts, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

3 2