- cabal-devel - Haskell.org

patch applied (hackage-server): "Allow building with ghc-7.0.x"
by devnull＠community.haskell.org 30 Nov '12

30 Nov '12

Fri Nov 30 22:55:25 GMT 2012 Duncan Coutts <duncan(a)well-typed.com> * Allow building with ghc-7.0.x Ignore-this: 4fd2e1ea85b072cbdaf99264b2f162c8 One of the servers I want to run it on only has ghc 7.0.4 M ./BuildClient.hs -2 +1 M ./Distribution/Server/Framework/Instances.hs +9 M ./hackage-server.cabal -4 +4

1 0

patch applied (hackage-server): "Add script to get all the old hackage data"
by devnull＠community.haskell.org 30 Nov '12

30 Nov '12

Fri Nov 30 17:46:53 GMT 2012 Duncan Coutts <duncan(a)well-typed.com> * Add script to get all the old hackage data Ignore-this: 513c7f1645078a8c7823e05e9b835ea7 ./hackage-import-script.sh -> ./old-hackage-import.sh A ./old-hackage-download.sh

1 0

patch applied (hackage-server): "Deal with restoring blobs generically" and 5 others
by devnull＠community.haskell.org 30 Nov '12

30 Nov '12

Fri Nov 30 11:00:21 GMT 2012 Edsko de Vries <edsko(a)well-typed.com> * Deal with restoring blobs generically Ignore-this: 28e695ea22637a21c1cf1af019df53e M ./Distribution/Server.hs -1 +2 M ./Distribution/Server/Features/BuildReports/Backup.hs -6 +9 M ./Distribution/Server/Features/Core/Backup.hs -17 +27 M ./Distribution/Server/Features/Distro/Backup.hs -4 +4 M ./Distribution/Server/Features/Documentation.hs -7 +6 M ./Distribution/Server/Features/DownloadCount/Backup.hs -2 +2 M ./Distribution/Server/Features/Tags/Backup.hs -1 +1 M ./Distribution/Server/Features/Upload/Backup.hs -2 +2 M ./Distribution/Server/Framework/BackupDump.hs -10 +27 M ./Distribution/Server/Framework/BackupRestore.hs -41 +105 M ./Distribution/Server/Users/Backup.hs -2 +2 M ./Main.hs -2 +4 Fri Nov 30 11:24:28 GMT 2012 Edsko de Vries <edsko(a)well-typed.com> * Use pure restore for download counts Ignore-this: 72ea2c139f2222f8d9bea49d52991630 M ./Distribution/Server/Features/DownloadCount/Backup.hs -19 +19 Fri Nov 30 16:02:21 GMT 2012 Edsko de Vries <edsko(a)well-typed.com> * Convert Core to use pure restore Ignore-this: a8596305889a168ac5304aa45a99772d In order to do this, define a Restore monad as follows: data Restore :: * -> * where RestoreDone :: forall a. a -> Restore a RestoreBind :: Restore a -> (a -> Restore b) -> Restore b RestoreFail :: forall a. String -> Restore a RestoreAddBlob :: ByteString -> Restore BlobId RestoreGetBlob :: BlobId -> Restore ByteString We could execute this in QuickCheck against a "fake" blob store (list of pairs of blob IDs and bytestrings) if we wish. M ./Distribution/Server/Features/Core/Backup.hs -92 +87 M ./Distribution/Server/Features/DownloadCount.hs -6 +6 M ./Distribution/Server/Features/DownloadCount/Backup.hs -5 +6 M ./Distribution/Server/Features/Tags.hs -6 +6 M ./Distribution/Server/Features/Tags/Backup.hs -7 +8 M ./Distribution/Server/Framework/BackupRestore.hs -15 +56 Fri Nov 30 16:25:14 GMT 2012 Edsko de Vries <edsko(a)well-typed.com> * Pure import for package maintainers Ignore-this: 9addb4e824c8e8754f5145ba167a44c0 M ./Distribution/Server/Features/Upload.hs -6 +6 M ./Distribution/Server/Features/Upload/Backup.hs -22 +25 Fri Nov 30 16:48:02 GMT 2012 Edsko de Vries <edsko(a)well-typed.com> * Update Documentation to use pure restore Ignore-this: 6f709e6e99abc08e14d36ccab1524d34 M ./Distribution/Server/Features/Documentation.hs -19 +25 M ./Distribution/Server/Util/ServeTarball.hs -5 +12 Fri Nov 30 17:35:51 GMT 2012 Edsko de Vries <edsko(a)well-typed.com> * Use pure import for Distro Ignore-this: d142a8b576a751c8351c27f9508b54bf M ./Distribution/Server/Features/Distro.hs -6 +6 M ./Distribution/Server/Features/Distro/Backup.hs -62 +49

1 0

patch applied (hackage-server): "Expose the mirrorers group resource" and 3 others
by devnull＠community.haskell.org 30 Nov '12

30 Nov '12

Fri Nov 30 17:10:15 GMT 2012 Duncan Coutts <duncan(a)well-typed.com> * Expose the mirrorers group resource Ignore-this: faa27af54953057bb3846b715fec7112 M ./Distribution/Server/Features/Mirror.hs +3 Fri Nov 30 17:10:51 GMT 2012 Duncan Coutts <duncan(a)well-typed.com> * Switch to digest auth by default Ignore-this: 313caf73aef646b40033e06dacbc948c M ./Distribution/Server/Framework/Auth.hs -2 +3 Fri Nov 30 17:34:00 GMT 2012 Duncan Coutts <duncan(a)well-typed.com> * Various minor tweaks to the import client Ignore-this: dff625c00be9c0283e9f549755e2a96e And rename the flag record fields in a consistent way. M ./ImportClient.hs -70 +97 Fri Nov 30 17:35:20 GMT 2012 Duncan Coutts <duncan(a)well-typed.com> * Add a script for importing all the old data from hackage Ignore-this: ef6a28770792f452db2fca096fb4aa0 A ./hackage-import-script.sh

1 0

Providing a smooth default user experience
by Johan Tibell 30 Nov '12

30 Nov '12

Hi all, Sensible defaults are important to provide a smooth user experience, especially for first-time users. There are a few areas where I think cabal could improve its default behavior. In particular, here are a couple of changes I would like to make: *cabal build should imply cabal install --only-dependencies * If you're currently working on a package, the right [1] way to build it is as follows: cabal install --only-dependencies cabal build (Note that in HEAD cabal build implies cabal configure so that step is no longer necessary, unless you need set some configure flags.) However, this is unintuitive and repetitive and many users end up using cabal install instead. The reasons I've heard for not having build imply installing dependencies is that users might not want packages to be downloaded from the internet, unless they explicitly use cabal install. While that's a fair point, I think it should be an opt-out rather than an opt-in because: - Users will end up have to install the dependencies anyway and there's typical no other options than installing them from Hackage. - Many users use cabal install instead of cabal build, for the reasons given above, making it a moot point. Proposal: add a --install-dependencies flag to cabal build that defaults to True. Once we start installing dependencies by default, the very first run of cabal build may take a long time, especially if sandboxing is used, so I think we should output the following message before installing dependencies: "Installing dependencies. This may take a long time if this is the first time you build this package." *Sandboxing should be used by default* This follows the "make it correct, then make it fast" principle. Sandboxing helps with a certain class of build problems, namely problems due to two different builds overwriting each others' dependencies in the shared package database. This problem shows up as a dangerously sounding cabal warning message, mentioning the --force-reinstalls flag. Sandboxing is an imperfect solution to this problem, requiring more rebuilding of packages than strictly necessary. A final solution would involve something like a Nix-style, append-only package database. The Nix solution is being worked on, as part of GSoC, but it's a difficult problem and it requires more wide-reaching changes (e.g. to ghc, ghc-pkg, ghci, and cabal). Since sandboxing is working today (with a few minor tweaks yet to be done) and it can be transparently replaced with a better solution once we have one, I think we should enable it by default in 1.18. Since sandboxing will result in more rebuilding, we should also enabled parallel builds by default, as they typically speed up builds by a factor of 2. Finally I think we need to improve messaging. We could use the same messaging as above i.e. "Installing dependencies. This may take a long time if this is the first time you build this package." Adding sandboxing by default also makes it easier to depend on unreleased versions of packages, as it has an add-source <dir> sub-command for managing those. *We should add a cabal run command* This proposal will only affect a small group of users, but will give those users a better user experience and hopefully encourage them to use cabal the right way. When developing e.g. web apps using Snap, the user needs to build and run an executable. You can do this today by running cabal build dist/build/my-exe/my-exe but e.g. the Snap documentation instead encourages users to do cabal install my-exe Perhaps because it's shorter and requires less explanation of where cabal puts its build output. I think we could improve this use case by introducing a new command, cabal run, with the following parameters: cabal run <executable> <args> which would run cabal build dist/build/<executable>/<executable> <args> Comments welcome! Cheers, Johan 1. The reason this is the right way is that you don't want to litter your shared package database with unreleased packages, as other packages might be built against these unreleased packages by misstake.

6 20

Custom preprocessor with custom file includes
by Jeroen Bransen 29 Nov '12

29 Nov '12

With Distribution.Simple.PreProcess.PreProcessor we can add a custom preprocessor to our Haskell project. This works fine when there is a one-to-one mapping from other-source to Haskell-source file, but there is no good support for other types of preprocessors. For example, for our UUAGC [1] project we also have a cabal plugin [2], and we have the following situation. In general, an AG file (.ag) is preprocessed into a single .hs file with the same name. However, AG files can include other AG files so typically multiple AG files are preprocessed in a single .hs file. Let's say we have a X.ag which includes Y.ag and Z.ag, and from this X.hs should be generated. The first time that we build this is fine, as cabal sees that it can generate X.hs from X.ag and it calls the custom preprocessor. However, if we now edit Y.ag and rebuild, nothing will happen! Cabal will only check the file time of X.ag, which is not changed, so decides that X.hs does not need to be re-generated. Currently we have solved this in a quite ugly way by means of a buildHook that removes the generated .hs file whenever this file should be rebuild. Although this works it's of course not a good solution, but with the current setup I see no other way of doing this. I believe this should be a more common situation and it should actually be up to the preprocessor to decide whether or not to rebuild a file. I suggest the following change to Distribution.Simple.PreProcess. In the current version, the check for rebuilding is in preprocessFile, which simply compares file change times of input/output files. I suggest to remove that check from there and always call the preprocessor. It is then up to the preprocessor to decide not to preprocess a file when the generated file is already in place and nothing has changed. To handle the common situation where there is a one-to-one mapping, I suggest to move this check to mkSimplePreprocessor which is probably used by almost all preprocessors. In that way nothing changes for 'simple' preprocessors, but there is some extra flexibility for the cases like UUAGC. Regards, Jeroen Bransen [1] http://hackage.haskell.org/package/uuagc [2] http://hackage.haskell.org/package/uuagc-cabal

2 2

patch applied (hackage-server): "Fix import for build reports"
by devnull＠community.haskell.org 29 Nov '12

29 Nov '12

Thu Nov 29 11:16:07 GMT 2012 Edsko de Vries <edsko(a)well-typed.com> * Fix import for build reports Ignore-this: c29ac5975354c117184b101e848272d4 Next up: try to do this in a pure fashion M ./Distribution/Server/Features/BuildReports/Backup.hs -2 +2

1 0

patch applied (hackage-server): "Fix comparison on tarballs"
by devnull＠community.haskell.org 29 Nov '12

29 Nov '12

Thu Nov 29 10:36:19 GMT 2012 Edsko de Vries <edsko(a)well-typed.com> * Fix comparison on tarballs Ignore-this: 648378d5c1cdccfb8db791fe524f64ce If the tarballs didn't contain the exact same entries, in the exact same order, then `equalTarBall` would report lots of superfluous errors due to a call to `mergeBy` on unsorted lists. M ./Distribution/Server/Framework/BackupRestore.hs -1 +3

1 0

Re: Cabal and GHC
by Simon Marlow 28 Nov '12

28 Nov '12

On 22/11/2012 19:54, Gábor Lehel wrote: > On Thu, Nov 22, 2012 at 8:41 PM, Henning Thielemann > <lemming(a)henning-thielemann.de> wrote: >> >> On Thu, 22 Nov 2012, Brandon Allbery wrote: >> >>> I am wondering if we're trying to solve the problem in the wrong way. The >>> core of the problem is that various things get baked into libraries in the >>> name of optimization, such that a given binary library has dependencies on >>> precise versions of other libraries; it's not like C where anything >>> supporting the ABI can use the same .a/.lib or .so/.dll/.dylib. >> >> >> C supports inlining. This should cause the same kind of problems, shouldn't >> it? > > Indeed, and in Qt for example which has a strict binary compatibility > guarantee, making a public function inline means you effectively > cannot change it until the next major release. (In this case "making > it inline" and "including the definition in the header file" are > effectively synonymous, one requires the other.) > > The problem is that in Haskell inlining is a lot more important for performance. I would like to see GHC support fixed ABIs, and the work I did with ABI hashing in GHC was aiming towards exactly that. For fixed ABIs you would need to have the user explicitly declare every inline function, and then hash the definitions as part of the ABI. (also do something about strictness and arity, and other cross-module optimisation hints). It might be painful, but it could be optional, and the gains are quite nice: the ability to upgrade a library in-place without recompiling everything that depends on it. Especially now that we're moving towards shared libraries, this would become more useful. I'm sure it's not going to happen soon, but it would be an interesting project for someone (probably larger than a GSoC project though). Cheers, Simon

3 2

patch applied (hackage-server): "Remove two old unused utils modules" and 5 others
by devnull＠community.haskell.org 27 Nov '12

27 Nov '12

Fri Nov 23 14:25:40 GMT 2012 Duncan Coutts <duncan(a)well-typed.com> * Remove two old unused utils modules Ignore-this: 15dcb6047f16fd05610359fac8ef0114 R ./Distribution/Server/Util/ActionLog.hs R ./Distribution/Server/Util/TimeLogger.hs Tue Nov 27 17:01:52 GMT 2012 Duncan Coutts <duncan(a)well-typed.com> * Simplify import client command line validation a bit Ignore-this: 2b30fc71f1896987368dcd68c667c028 M ./ImportClient.hs -13 +14 Tue Nov 27 17:05:47 GMT 2012 Duncan Coutts <duncan(a)well-typed.com> * Allow concurrent upload for package metadata in the import client Ignore-this: e9656155c4f6973914347da92be4a1b0 M ./ImportClient.hs -16 +33 Tue Nov 27 16:57:28 GMT 2012 Duncan Coutts <duncan(a)well-typed.com> * Grrr, the periodic download refresh is unconditional Ignore-this: 5698c5c4f327723da89d8263e2a2c6d9 In the meantime, make it wait a bit longer. M ./Distribution/Server/Features/PackageList.hs -1 +2 Tue Nov 27 17:30:59 GMT 2012 Duncan Coutts <duncan(a)well-typed.com> * Overhaul all the caching Ignore-this: e82f108830a5b42576bd4b0bba677ed4 Separate the previous uses of caches into two kinds: 1. simple in-memory vars and 2. proper caches. For the former we just add a simple strict wrapper around MVar, with read/write/modify operations, while for latter kind we use asynchronous caches that are constructed with their update action. Then update the features to use the appropriate kind of cache. Additionally, the old async cache infrastructure had a feature to disable updating of caches for the benefit of bulk import operations. The corresponding thing in the new infrastructure is a tunable knob to delay cache updates so that more changes can accumulate before doing the expensive cache update operation. M ./Distribution/Server.hs -6 +9 M ./Distribution/Server/Features.hs -5 +5 M ./Distribution/Server/Features/Core.hs -29 +50 M ./Distribution/Server/Features/DownloadCount.hs -7 +6 M ./Distribution/Server/Features/Html.hs -29 +40 M ./Distribution/Server/Features/NameSearch.hs -30 +63 M ./Distribution/Server/Features/PackageList.hs -13 +12 M ./Distribution/Server/Features/Packages.hs -14 +18 M ./Distribution/Server/Features/Tags.hs -5 +4 M ./Distribution/Server/Features/Users.hs -12 +11 M ./Distribution/Server/Framework.hs -2 +6 M ./Distribution/Server/Framework/Cache.hs -79 +129 M ./Distribution/Server/Framework/Instances.hs -1 +8 A ./Distribution/Server/Framework/MemState.hs M ./Distribution/Server/Framework/Types.hs -1 +12 R ./Distribution/Server/Util/AsyncVar.hs M ./Distribution/Server/Util/TextSearch.hs -2 +5 M ./Main.hs -18 +26 M ./hackage-server.cabal -1 +1 Tue Nov 27 17:41:01 GMT 2012 Duncan Coutts <duncan(a)well-typed.com> * Move a use of runHook into the feature that owns the hook Ignore-this: ffaef756e8b9c49027a4cf867c64d0be Other features should not really prod hooks. M ./Distribution/Server/Features/Core.hs +1 M ./Distribution/Server/Features/PreferredVersions.hs -1

1 0