Re: [Haskell-cafe] Hackage inconsistent? (pandoc.cabal in 1.13.1)

On 10/27/2014 07:48 PM, Erik Hesselink wrote:

I was a bit hesitant about using this feature as well. We've had some confusion when we used it [1][2][3]. The Nix guys also seem to be having some trouble fitting it into their workflow [4]. But in [1] I did a quick grep to see how many packages used the feature, and there are already quite a lot. There is an 'x-revision' property added to the cabal file in the index, so in theory I guess tools could learn to deal with it.

In general I think the biggest win for this feature is not even in relaxing bounds, but in tightening them when it turns out they're too loose. Previously cabal was always free to pick an old, unconstrained version and would often do so, leading to type errors during builds.

Erik

[1] https://github.com/silkapp/rest/issues/57 [2] https://github.com/silkapp/aeson-utils/issues/1 [3] https://github.com/silkapp/json-schema/issues/12 [4] https://github.com/NixOS/cabal2nix/issues/84

On Mon, Oct 27, 2014 at 8:30 PM, John MacFarlane wrote:

...

I must say, I did pause a bit before making this change to pandoc on Hackage, precisely because I realized that the cabal file in the package would not match the one on Hackage.

But I went ahead, assuming that if the means were provided to make this kind of change, it should be okay.

It is time-consuming for me to make a proper pandoc release. I need to write up release notes, rebuild the website, generate and test the binary packages on various different platforms, upload to github releases and Hackage, send announcement emails, etc. Even with automation this all takes work. I'd rather not have to do this every time a version bound changes on a dependent package, so I like this way of making minor tweaks to version bounds; I hope I can be persuaded that it's not a bad idea.

John

TL;DR version of below: I think Hackage should generate a new package version for each info change rather than the change being ‘silent’ making it pretty exclusive to tools which don't care about what they actually build and what hash it has (cabal). This brings us back to ‘all version are explicit’ from the ‘we have versions but there are some updates you need to pull in your tools too’. Long version: As Erik points out at his [4], this really messes things up for us in the nix world, and really anywhere that 1. Wants to grab from Hackage 2. Wants a reproducible build The common justification for this is that it doesn't change the tarball hash. Well then, this ‘update info after upload’ feature then automatically excludes every single system which uses this tarball. Why? 1. Maintainer uploads a package 2. Distros package it and note down the hash 3. Something happens and maintainer bumps the bounds on Hackage only 4. Distros still grab the very same package as they did before (tarball doesn't change, remember?). Inside is the cabal file with old dependencies, so the build fails if the distro updated other packages (almost certainly). So even though the hash doesn't change, builds suddenly start failing. If you're using cabal, you don't get a reproducible build at all because which versions it picks depends on whatever info is on Hackage at build time (well, last cabal update time) and not what's in the tarball. It is currently perfectly possible for two separate users to have ‘cabal update’ and know about exactly the same package versions available but end up with different version numbers used because one ‘cabal update’d before the info on Hackage was changed and the other did not. But John is right that sometimes making releases is tiresome. For version bumps it can be as easy as checking out the last tag, bumping deps and uploading, but it can be much more involved too. I admit that I used this myself, although only for the stuff that hope to release properly soon and I happen to maintain on NixOS too which AFAICT is the only distro which packages Haskell fast enough to notice breakage within few days (in huge part thanks to Peter Simons ;) ) So clearly there is a problem here: only cabal users use the information from Hackage info bumps. Everyone else who looks into the tarball just fails to build. cabal users who happen to cabal update at unfortunate time also fail to build. I can only think of one ‘solution’ which fixes this and doesn't require considerable effort and co-ordination between maintainers and packagers. Here is how situation currently looks like 1. I upload awesomepackage (ap) to Hackage under ap-1.0.0.0 2. Time passes, I have commits on top 3. One of those packager type of folk who like to demand things come onto my issue tracker and say I should update my dependencies but I don't want to release right now 4. I go onto Hackage and use that new cool feature, changing the cabal file on ap-1.0.0.0. 5. Packager is unhappy because the problem is not fixed, ap-1.0.0.0 gives them the same thing. The only thing they can do is monkeypatch deps on their side. Instead, I think at steps 4 and 5 the following should happen: 4. I make changes to ap-1.0.0.0, Hackage generates ap-1.0.0.0-fix1 package with identical content but new info without any extra effort on my side. 5. Packager simply now points to ap-1.0.0.0-fix1 which has the new hash and builds great again. cabal does The Right Thing™ here and resolves ap-1.0.0.0 to ap-1.0.0.0-fix1 so everything keeps working there as it does now. It does seem a bit wasteful space-wise but with clever programming on Hackage side that could probably be avoided. There is no longer a problem with these ‘hidden’ changes happening. Everyone can point at the *exact* version they want used like they do now and they get what exactly what they asked for. A less wasteful way would be for the packaging distros to update their tools to look at the Hackage-only cabal files as well. Personally I think that seems like the wrong place to solve this though. Lastly, Hackage should announce with big fat red letters on the front page of the package that the source tarball contains different info than what's shown on Hackage. It's really not possible to tell as a user at the moment unless we specifically look for it. Sorry for the long-winded message, hopefully you heeded the TL;DR at the top and skipped this. -- Mateusz K.