I must say, I did pause a bit before making this change to pandoc on Hackage, precisely because I realized that the cabal file in the package would not match the one on Hackage. But I went ahead, assuming that if the means were provided to make this kind of change, it should be okay. It is time-consuming for me to make a proper pandoc release. I need to write up release notes, rebuild the website, generate and test the binary packages on various different platforms, upload to github releases and Hackage, send announcement emails, etc. Even with automation this all takes work. I'd rather not have to do this every time a version bound changes on a dependent package, so I like this way of making minor tweaks to version bounds; I hope I can be persuaded that it's not a bad idea. John

On Mon, Oct 27, 2014 at 08:48:16PM +0100, Erik Hesselink wrote:

I was a bit hesitant about using this feature as well. We've had some confusion when we used it [1][2][3]. The Nix guys also seem to be having some trouble fitting it into their workflow [4]. But in [1] I did a quick grep to see how many packages used the feature, and there are already quite a lot. There is an 'x-revision' property added to the cabal file in the index, so in theory I guess tools could learn to deal with it.

Personally I would have liked the feature more if a change to tools had been required in order to take advantage of the modified .cabal.

In general I think the biggest win for this feature is not even in relaxing bounds, but in tightening them when it turns out they're too loose. Previously cabal was always free to pick an old, unconstrained version and would often do so, leading to type errors during builds.

I gather this means that cabal-install has been modified to (or maybe it always has) replace the in-tar-ball .cabal with the .cabal from the 00-index. Is that correct? /M -- Magnus Therning OpenPGP: 0xAB4DFBA4 email: magnus@therning.org jabber: magnus@therning.org twitter: magthe http://therning.org/magnus In a hierarchy, every employee tends to rise to his level of incompetence. -- The Peter Principle

On 10/27/2014 07:48 PM, Erik Hesselink wrote:

I was a bit hesitant about using this feature as well. We've had some confusion when we used it [1][2][3]. The Nix guys also seem to be having some trouble fitting it into their workflow [4]. But in [1] I did a quick grep to see how many packages used the feature, and there are already quite a lot. There is an 'x-revision' property added to the cabal file in the index, so in theory I guess tools could learn to deal with it.

In general I think the biggest win for this feature is not even in relaxing bounds, but in tightening them when it turns out they're too loose. Previously cabal was always free to pick an old, unconstrained version and would often do so, leading to type errors during builds.

Erik

[1] https://github.com/silkapp/rest/issues/57 [2] https://github.com/silkapp/aeson-utils/issues/1 [3] https://github.com/silkapp/json-schema/issues/12 [4] https://github.com/NixOS/cabal2nix/issues/84

On Mon, Oct 27, 2014 at 8:30 PM, John MacFarlane wrote:

...

I must say, I did pause a bit before making this change to pandoc on Hackage, precisely because I realized that the cabal file in the package would not match the one on Hackage.

But I went ahead, assuming that if the means were provided to make this kind of change, it should be okay.

It is time-consuming for me to make a proper pandoc release. I need to write up release notes, rebuild the website, generate and test the binary packages on various different platforms, upload to github releases and Hackage, send announcement emails, etc. Even with automation this all takes work. I'd rather not have to do this every time a version bound changes on a dependent package, so I like this way of making minor tweaks to version bounds; I hope I can be persuaded that it's not a bad idea.

John

TL;DR version of below: I think Hackage should generate a new package version for each info change rather than the change being ‘silent’ making it pretty exclusive to tools which don't care about what they actually build and what hash it has (cabal). This brings us back to ‘all version are explicit’ from the ‘we have versions but there are some updates you need to pull in your tools too’. Long version: As Erik points out at his [4], this really messes things up for us in the nix world, and really anywhere that 1. Wants to grab from Hackage 2. Wants a reproducible build The common justification for this is that it doesn't change the tarball hash. Well then, this ‘update info after upload’ feature then automatically excludes every single system which uses this tarball. Why? 1. Maintainer uploads a package 2. Distros package it and note down the hash 3. Something happens and maintainer bumps the bounds on Hackage only 4. Distros still grab the very same package as they did before (tarball doesn't change, remember?). Inside is the cabal file with old dependencies, so the build fails if the distro updated other packages (almost certainly). So even though the hash doesn't change, builds suddenly start failing. If you're using cabal, you don't get a reproducible build at all because which versions it picks depends on whatever info is on Hackage at build time (well, last cabal update time) and not what's in the tarball. It is currently perfectly possible for two separate users to have ‘cabal update’ and know about exactly the same package versions available but end up with different version numbers used because one ‘cabal update’d before the info on Hackage was changed and the other did not. But John is right that sometimes making releases is tiresome. For version bumps it can be as easy as checking out the last tag, bumping deps and uploading, but it can be much more involved too. I admit that I used this myself, although only for the stuff that hope to release properly soon and I happen to maintain on NixOS too which AFAICT is the only distro which packages Haskell fast enough to notice breakage within few days (in huge part thanks to Peter Simons ;) ) So clearly there is a problem here: only cabal users use the information from Hackage info bumps. Everyone else who looks into the tarball just fails to build. cabal users who happen to cabal update at unfortunate time also fail to build. I can only think of one ‘solution’ which fixes this and doesn't require considerable effort and co-ordination between maintainers and packagers. Here is how situation currently looks like 1. I upload awesomepackage (ap) to Hackage under ap-1.0.0.0 2. Time passes, I have commits on top 3. One of those packager type of folk who like to demand things come onto my issue tracker and say I should update my dependencies but I don't want to release right now 4. I go onto Hackage and use that new cool feature, changing the cabal file on ap-1.0.0.0. 5. Packager is unhappy because the problem is not fixed, ap-1.0.0.0 gives them the same thing. The only thing they can do is monkeypatch deps on their side. Instead, I think at steps 4 and 5 the following should happen: 4. I make changes to ap-1.0.0.0, Hackage generates ap-1.0.0.0-fix1 package with identical content but new info without any extra effort on my side. 5. Packager simply now points to ap-1.0.0.0-fix1 which has the new hash and builds great again. cabal does The Right Thing™ here and resolves ap-1.0.0.0 to ap-1.0.0.0-fix1 so everything keeps working there as it does now. It does seem a bit wasteful space-wise but with clever programming on Hackage side that could probably be avoided. There is no longer a problem with these ‘hidden’ changes happening. Everyone can point at the *exact* version they want used like they do now and they get what exactly what they asked for. A less wasteful way would be for the packaging distros to update their tools to look at the Hackage-only cabal files as well. Personally I think that seems like the wrong place to solve this though. Lastly, Hackage should announce with big fat red letters on the front page of the package that the source tarball contains different info than what's shown on Hackage. It's really not possible to tell as a user at the moment unless we specifically look for it. Sorry for the long-winded message, hopefully you heeded the TL;DR at the top and skipped this. -- Mateusz K.

per se, thats what hackage DOES now. Though it is valid to want some sort of +patchlevel suffix to be visible for distro engineering purposes or such. Speaking as a hackage trustee, this .cabal editing facility has been a VERY VERY good thing overall. A lot of problems have been prevented because of it, and that its creating new problems that haven't existed before should be considered a testament to its success! ;) On Tue, Oct 28, 2014 at 8:17 AM, Andreas Abel wrote:

On 28.10.2014 01:38, Mateusz Kowalczyk wrote:

...

4. I make changes to ap-1.0.0.0, Hackage generates ap-1.0.0.0-fix1 package with identical content but new info without any extra effort on my side.

Basically you suggest that there should be a last digit in the version number that is only used for dependency fixes and is handled automatically by cabal and hackage. We could as well say that Hackage generates

ap-1.0.0.0.1 ap-1.0.0.0.2 -- for the next dependency fix

but you cannot specify the 5th digit in cabal commands, so when you say

cabal install ap-1.0.0.0

you get the latest, i.e. ap-1.0.0.0.2.

--Andreas

On 28.10.2014 01:38, Mateusz Kowalczyk wrote:

...

On 10/27/2014 07:48 PM, Erik Hesselink wrote:

...

I was a bit hesitant about using this feature as well. We've had some confusion when we used it [1][2][3]. The Nix guys also seem to be having some trouble fitting it into their workflow [4]. But in [1] I did a quick grep to see how many packages used the feature, and there are already quite a lot. There is an 'x-revision' property added to the cabal file in the index, so in theory I guess tools could learn to deal with it.

In general I think the biggest win for this feature is not even in relaxing bounds, but in tightening them when it turns out they're too loose. Previously cabal was always free to pick an old, unconstrained version and would often do so, leading to type errors during builds.

Erik

[1] https://github.com/silkapp/rest/issues/57 [2] https://github.com/silkapp/aeson-utils/issues/1 [3] https://github.com/silkapp/json-schema/issues/12 [4] https://github.com/NixOS/cabal2nix/issues/84

On Mon, Oct 27, 2014 at 8:30 PM, John MacFarlane wrote:

...

I must say, I did pause a bit before making this change to pandoc on Hackage, precisely because I realized that the cabal file in the package would not match the one on Hackage.

But I went ahead, assuming that if the means were provided to make this kind of change, it should be okay.

It is time-consuming for me to make a proper pandoc release. I need to write up release notes, rebuild the website, generate and test the binary packages on various different platforms, upload to github releases and Hackage, send announcement emails, etc. Even with automation this all takes work. I'd rather not have to do this every time a version bound changes on a dependent package, so I like this way of making minor tweaks to version bounds; I hope I can be persuaded that it's not a bad idea.

John

TL;DR version of below: I think Hackage should generate a new package version for each info change rather than the change being ‘silent’ making it pretty exclusive to tools which don't care about what they actually build and what hash it has (cabal). This brings us back to ‘all version are explicit’ from the ‘we have versions but there are some updates you need to pull in your tools too’.

Long version:

As Erik points out at his [4], this really messes things up for us in the nix world, and really anywhere that

1. Wants to grab from Hackage 2. Wants a reproducible build

The common justification for this is that it doesn't change the tarball hash. Well then, this ‘update info after upload’ feature then automatically excludes every single system which uses this tarball. Why?

1. Maintainer uploads a package 2. Distros package it and note down the hash 3. Something happens and maintainer bumps the bounds on Hackage only 4. Distros still grab the very same package as they did before (tarball doesn't change, remember?). Inside is the cabal file with old dependencies, so the build fails if the distro updated other packages (almost certainly).

So even though the hash doesn't change, builds suddenly start failing. If you're using cabal, you don't get a reproducible build at all because which versions it picks depends on whatever info is on Hackage at build time (well, last cabal update time) and not what's in the tarball. It is currently perfectly possible for two separate users to have ‘cabal update’ and know about exactly the same package versions available but end up with different version numbers used because one ‘cabal update’d before the info on Hackage was changed and the other did not.

But John is right that sometimes making releases is tiresome. For version bumps it can be as easy as checking out the last tag, bumping deps and uploading, but it can be much more involved too. I admit that I used this myself, although only for the stuff that hope to release properly soon and I happen to maintain on NixOS too which AFAICT is the only distro which packages Haskell fast enough to notice breakage within few days (in huge part thanks to Peter Simons ;) )

So clearly there is a problem here: only cabal users use the information from Hackage info bumps. Everyone else who looks into the tarball just fails to build. cabal users who happen to cabal update at unfortunate time also fail to build.

I can only think of one ‘solution’ which fixes this and doesn't require considerable effort and co-ordination between maintainers and packagers. Here is how situation currently looks like

1. I upload awesomepackage (ap) to Hackage under ap-1.0.0.0 2. Time passes, I have commits on top 3. One of those packager type of folk who like to demand things come onto my issue tracker and say I should update my dependencies but I don't want to release right now 4. I go onto Hackage and use that new cool feature, changing the cabal file on ap-1.0.0.0. 5. Packager is unhappy because the problem is not fixed, ap-1.0.0.0 gives them the same thing. The only thing they can do is monkeypatch deps on their side.

Instead, I think at steps 4 and 5 the following should happen:

4. I make changes to ap-1.0.0.0, Hackage generates ap-1.0.0.0-fix1 package with identical content but new info without any extra effort on my side. 5. Packager simply now points to ap-1.0.0.0-fix1 which has the new hash and builds great again. cabal does The Right Thing™ here and resolves ap-1.0.0.0 to ap-1.0.0.0-fix1 so everything keeps working there as it does now.

It does seem a bit wasteful space-wise but with clever programming on Hackage side that could probably be avoided. There is no longer a problem with these ‘hidden’ changes happening. Everyone can point at the *exact* version they want used like they do now and they get what exactly what they asked for. A less wasteful way would be for the packaging distros to update their tools to look at the Hackage-only cabal files as well. Personally I think that seems like the wrong place to solve this though.

Lastly, Hackage should announce with big fat red letters on the front page of the package that the source tarball contains different info than what's shown on Hackage. It's really not possible to tell as a user at the moment unless we specifically look for it.

Sorry for the long-winded message, hopefully you heeded the TL;DR at the top and skipped this.

-- Andreas Abel <>< Du bist der geliebte Mensch.

Department of Computer Science and Engineering Chalmers and Gothenburg University, Sweden

andreas.abel@gu.se http://www2.tcs.ifi.lmu.de/~abel/

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Hi, On 29 October 2014 08:40, Magnus Therning wrote:

I'm not a cabal-install user, so I'm curious to hear how this change works with cabal-install 1.18.0.5, which is what we package in ArchHaskell. Any problems with that? If so, will support for this feature be backported?

This feature was added two years ago, in the 1.16 release [1]. [1] https://github.com/haskell/cabal/commit/22d3772222ebb5dd49d60882845915999ebc...

Hi Andreas,

On 28.10.2014 01:38, Mateusz Kowalczyk wrote:

...

4. I make changes to ap-1.0.0.0, Hackage generates ap-1.0.0.0-fix1 package with identical content but new info without any extra effort on my side.

Basically you suggest that there should be a last digit in the version number that is only used for dependency fixes and is handled automatically by cabal and hackage.

the important part of Mateusz suggestion is that the version number ought to *change* after an edit. The exact mechanics of that change -- i.e. whether Hackage bumps some digit or appends a tag or whatever -- aren't very important. What is important is that it's possible to distinguish the edited version from the original one. As of now, users cannot tell which variant of the package they've installed.

We could as well say that Hackage generates

ap-1.0.0.0.1 ap-1.0.0.0.2 -- for the next dependency fix

That would be one possible solution, yes. Personally, I think these kind of version numbers are a bit silly, so I would prefer to append a tag, but I don't feel strongly about that.

You cannot specify the 5th digit in cabal commands, so when you say

cabal install ap-1.0.0.0

you get the latest, i.e. ap-1.0.0.0.2.

Hackage edits are destructive: the original version is gone afterwards, and it's no longer possible to install it. As such, this point doesn't seem relevant. What is relevant, however, that people who have installed version ap-1.0.0.0.1 prior to the edit can actually tell that there is an update available -- ap-1.0.0.0.2 is *newer*. In the current state of affairs, it's not possible to tell that a package has been updated ("edited") just by looking at its version number. Best regards, Peter

2014-10-30 12:08 GMT+01:00 Tom Ellis :

On Thu, Oct 30, 2014 at 12:01:09PM +0100, Peter Simons wrote:

...

...

We could as well say that Hackage generates

ap-1.0.0.0.1 ap-1.0.0.0.2 -- for the next dependency fix

That would be one possible solution, yes. Personally, I think these kind of version numbers are a bit silly, so I would prefer to append a tag, but I don't feel strongly about that.

How would this be any different from just uploading a new version with an updated cabal file?

From a user perspective it wouldn't be different and it shouldn't be. From a maintainer/trustee POV it's just a convenience feature, which is OK. Having said that, I think that automatically bumping the minor version number would be the right thing to do. Breaking reproducible builds is a horrible idea.

2014-10-30 13:40 GMT+01:00 Ivan Lazar Miljenovic :

I don't think you should bump the minor version (as in the fourth version field) as it doesn't affect any actual changes that may have taken place in SCM, etc. [...]

Fiddling around with anything which affects compilation in which the package in question involved *must* be reflected somehow in the version number, otherwise we could just live without any versions at all. If we agree that automatic version bumps are not what we want, we should disable editing the .cabal file on Hackage completely. As it is, people with identical output of "ghc-pkg list" could have completely different experiences when they try to compile the same thing, which is horrible.

Hi Lennart,

Bumping [the (n+1)th digit] is not safe, as the maintainer might decide to publish a new version that adds a new component.)

Suppose I publish foo-1.0. Then a sequence of edits would produce versions 1.0.1, 1.0.1.1, 1.0.1.1.1, and so on. Why would there be any problem? Best regards, Peter

now that you remind me: may i make a related suggestion to change the wording in the explanation of the lexicographic ordering: Instead of "For example, 2.1 > 1.3, and 2.1.1 > 2.1." Write "For example, 2.0.0 > 1.3.2, and 2.0.1.0 > 2.0.1." for two reasons 1) an example following the A.B.C definition should have three components (even if we just define "lexicographical" independent of number-of-components; it is mistakable) 2) i once assumed that a.b == a.b.0 ; this change would prevent such. (Or is there any reason against using trailing zeros?)

On 30 October 2014 22:09, Erik Hesselink wrote:

On Thu, Oct 30, 2014 at 12:01 PM, Peter Simons wrote:

...

aren't very important. What is important is that it's possible to distinguish the edited version from the original one. As of now, users cannot tell which variant of the package they've installed.

Why is this important? I believe there is no difference between two revisions after they're installed, at least not currently, since you're only allowed to edit dependency version ranges, and perhaps some informational fields.

* For people doing cabal -> foo converters to know that there _is_ a newer version * "Oh, wait, I can't update the version of library foo because I've got the latest available version of bar and it doesn't support the new version of foo" without knowing there's a (pseudo-) newer version which _does_ allow the newer version of foo.

Erik _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

-- Ivan Lazar Miljenovic Ivan.Miljenovic@gmail.com http://IvanMiljenovic.wordpress.com

Hi, On 30 October 2014 12:01, Peter Simons wrote:

In the current state of affairs, it's not possible to tell that a package has been updated ("edited") just by looking at its version number.

There is a 'x-hackage-revision' field in the .cabal file which gets updated each time the package description is edited. May be useful for generating distribution packages, e.g. you could have foo-1.0, foo-1.0-rev1, foo-1.0-rev2, etc. Regarding version tags - they're deprecated and are going to be removed in the 7.12 release of GHC.

On Mon, Oct 27, 2014 at 12:30:03PM -0700, John MacFarlane wrote:

I must say, I did pause a bit before making this change to pandoc on Hackage, precisely because I realized that the cabal file in the package would not match the one on Hackage.

But I went ahead, assuming that if the means were provided to make this kind of change, it should be okay.

It is time-consuming for me to make a proper pandoc release. I need to write up release notes, rebuild the website, generate and test the binary packages on various different platforms, upload to github releases and Hackage, send announcement emails, etc. Even with automation this all takes work. I'd rather not have to do this every time a version bound changes on a dependent package, so I like this way of making minor tweaks to version bounds; I hope I can be persuaded that it's not a bad idea.

I can fully appreciate the usefulness of the feature; most of the patches I make to package Haskell packages for Archlinux is of the kind that modifies dependency restrictions. I've read through the discussion at https://github.com/haskell/hackage-server/issues/52 but I see no one arguing for making this change a bit less intrusive and more opt-in for tools, e.g. by keeping 00-index as it's always been and then putting the Hackage edits of .cabals in an overlay (an 01-index). Has this been up for proposed/discussed somewhere else? /M -- Magnus Therning OpenPGP: 0xAB4DFBA4 email: magnus@therning.org jabber: magnus@therning.org twitter: magthe http://therning.org/magnus As long as there are ill-defined goals, bizarre bugs, and unrealistic schedules, there will be Real Programmers willing to jump in and Solve The Problem, saving the documentation for later. Long live Fortran! -- Ed Post