2009/1/30 Krzysztof Skrzętnicki

On Sat, Jan 31, 2009 at 00:07, Valentyn Kamyshenko wrote:

...

Hello John,

I'm trying to install the new version of HDBC on my Mac (ghc version is 6.10.1), and get the following error:

# cabal upgrade --global Resolving dependencies... 'HDBC-2.0.0' is cached. Configuring HDBC-2.0.0... Preprocessing library HDBC-2.0.0... Preprocessing executables for HDBC-2.0.0... Building HDBC-2.0.0... [1 of 7] Compiling Database.HDBC.SqlValue ( Database/HDBC/SqlValue.hs, dist/build/Database/HDBC/SqlValue.o ) [2 of 7] Compiling Database.HDBC.ColTypes ( Database/HDBC/ColTypes.hs, dist/build/Database/HDBC/ColTypes.o ) [3 of 7] Compiling Database.HDBC.Statement ( Database/HDBC/Statement.hs, dist/build/Database/HDBC/Statement.o )

Database/HDBC/Statement.hs:113:9: Type constructor `Exception' used as a class In the instance declaration for `Exception SqlError' cabal: Error: some packages failed to install: HDBC-2.0.0 failed during the building phase. The exception was: exit: ExitFailure 1

I got the same error: Windows XP, GHC 6.10.1. For some reason imported Control.Exception module appears to be 6.8.* version.

I got that error when using cabal to install. However, when I downloaded the package and installed it with a ./Setup.lhs configure..., it worked. Maybe this is a cabal issue instead. Michael

On Fri, 2009-01-30 at 18:29 -0600, John Goerzen wrote:

On Fri, Jan 30, 2009 at 03:50:30PM -0800, Michael Snoyman wrote:

...

...

...

[3 of 7] Compiling Database.HDBC.Statement ( Database/HDBC/Statement.hs, dist/build/Database/HDBC/Statement.o )

Database/HDBC/Statement.hs:113:9: Type constructor `Exception' used as a class In the instance declaration for `Exception SqlError' cabal: Error: some packages failed to install: HDBC-2.0.0 failed during the building phase. The exception was: exit: ExitFailure 1

That's *WEIRD*.

It's as if you have the old base from GHC 6.8. Is cabal-install doing something (weird|evil|smart) here?

Yes.

Leads me to think even more that there's cabal-install trickery here.

Yup.

Can someone enlighten us on that? Is cabal-install playing tricks with base?

Yes. The difference (in the released version of cabal-install) between the configure and install commands is that configure uses a dumb algorithm for deciding what versions of dependencies to use while install uses a smarter algorithm. Obviously this is confusing and in the next release configure will use the smart algorithm too. At the moment configure picks the latest versions of all packages irrespective of whether this is likely to work or not. For developers using ghc-6.10 with base 3 and 4 this means they end up with base 4. They then likely do not notice that their package actually needs base version 4. The install command uses the constraint solver and takes into account some global preferences from hackage. These are soft preferences / soft constraints. They do not override constraints specified in the .cabal file or on the command line. However for the huge number of packages on hackage that worked with ghc-6.8 but failed to specify a constraint on the version of base, using a preference of base 3 over base 4 enables those packages to continue to build. So in the next cabal-install release (which should be pretty soon now) configure will do the same thing and pick base 3 unless you specify build-depends base >= 4. So the solution is for HDBC-2.0 to specify the version of base that it needs more accurately. It appears that it does need version 4 so it should use: build-depends: base == 4.* On a similar issue, I am going to make Hackage enforce that packages specify an upper bounds on the version of base. Hopefully doing that will make people also consider the appropriate lower bound. I don't see an obvious way of automatically requiring accuracy in selecting the base version between 3 and 4. It is perfectly possible to have a package that works with both. Perhaps we should make not-specifying an upper bound an error and not specifying a lower bound a warning. Thoughts? Duncan

On Sun, 2009-02-01 at 15:56 +0100, Niklas Broberg wrote:

...

So in the next cabal-install release (which should be pretty soon now) configure will do the same thing and pick base 3 unless you specify build-depends base >= 4.

... and so there will never be any incentive for these many packages to migrate to base-4, which also has consequences for packages that do want to use base-4, but also want to depend on such packages. And so base-3 will live on in eternity, and there was never any point in doing that new base release at all.

I really really think this is the wrong way to go. Occasional destruction is desperately needed for progress, else things will invariably stagnate.

I disagree. Having everything fail (we measured it as ~90% of hackage) when people upgraded to ghc-6.10 would have been a disaster. Do you recall the screaming, wailing and gnashing of teeth after the release of ghc-6.8 when most of hackage broke? We (I mean ghc and cabal hackers) got a lot of flak for not making the upgrade process easier and needlessly breaking everyone's perfectly good packages. This time round we went to a lot of effort to make the upgrade process smooth. And for the most part it was. Only a small proportion of hackage packages broke. Now I agree that there is a problem with new packages where the configure selects base 4 but install selects base 3. I've improved that in the darcs version. You're also right that during the lifespan of base 4 we need to encourage new releases to start working with it because we cannot stick with base 3 for ever. Doing that with warnings hints etc is the way to go. Destruction is not such a friendly approach. We do not need to make the users suffer, we just need to inform and persuade developers uploading new releases to do the right thing. Duncan

On Sun, 2009-02-01 at 17:22 +0100, Niklas Broberg wrote:

...

...

I really really think this is the wrong way to go. Occasional destruction is desperately needed for progress, else things will invariably stagnate.

I disagree. Having everything fail (we measured it as ~90% of hackage) when people upgraded to ghc-6.10 would have been a disaster. Do you recall the screaming, wailing and gnashing of teeth after the release of ghc-6.8 when most of hackage broke? We (I mean ghc and cabal hackers) got a lot of flak for not making the upgrade process easier and needlessly breaking everyone's perfectly good packages.

90%? That sounds like an awful lot for the relatively minor changes with base-4. I guess a number of packages will use exceptions, and others with use Data and Typeable, but 90%? I don't doubt you when you say that you measured it though, just marvelling.

http://www.haskell.org/pipermail/glasgow-haskell-users/2008-October/015654.h... Most non-trivial packages either use exceptions or depend on a package that uses exceptions (or syb). The 90% is from memory however. I can't find our exact measurements. I think we may have given up trying to measure precisely after it became obvious that nothing was working.

If 90% of hackage would truly break, then I agree that we might need more caution than my radical approach. But I'm not fully convinced there either.

I think in the end we managed to get it down to around 5% breakage which I felt was a pretty good result.

After all, unlike with the ghc-6.8 release, all that's needed for a package to work again is to upload a new .cabal that makes the dependency on base-3 explicit,

For the most part that was all that was required for 6.8 too, to add dependencies on the new packages split out of base. People still complained. A lot. :-)

if an author really doesn't want to update the codebase. And even for those packages where library authors don't do that simple step, all that's needed of the user of the library is to specify the base-3 constraint when running cabal-install.

But people do not know that. It has to work by default (even if it warns).

My main complaint is really that there is currently no incentive whatsoever for library authors to migrate. If we make base-4 the default, it will require just a little bit of work to make packages that depend on base-3 work anyway, as seen above. It's not so much work that it should incur any screaming, wailing and teeth gnashing. But it should be just enough work to encourage action in one way or another, either truly migrating the code or just making the dependency explicit as I noted above. I think it would be hard to find a more accurate, and non-intrusive, incentive. :-)

I think the right place to put incentives is in checks in new uploads to hackage. Making existing packages break should be avoided when possible, even when the changes the developer needs to make are trivial. Many many end users have no idea how to make these trivial changes and it is not our intention to punish them. From the perspective of an end user just trying to install something it either works or it doesn't. When a large proportion stop working is when the teeth gnashing starts. Also remember that many packages do not have very active maintainers.

I definitely agree with your suggestion to make hackage require an upper bound on base. But that's to make us future proof,

True.

it won't solve the issue here and now.

What do you think about the suggestions in my other reply? I hope that is a reasonable suggestion to encourage a transition to base 4 without punishing end users. Duncan

On Mon, 2009-02-02 at 15:22 +0200, Yitzchak Gale wrote:

Duncan Coutts wrote:

...

...

...

So in the next cabal-install release (which should be pretty soon now) configure will do the same thing and pick base 3 unless you specify build-depends base >= 4.

Niklas Broberg wrote:

...

...

I really really think this is the wrong way to go. Occasional destruction is desperately needed for progress, else things will invariably stagnate.

...

I disagree. Having everything fail... would have been a disaster... during the lifespan of base 4 we need to encourage new releases to start working with it... Doing that with warnings hints etc is the way to go.

No, that's not good enough either. Existing packages will just stay with old versions to avoid the work, and new packages will then also use old versions for maximum compatibility.

The incentive is not strong enough. Those warnings and hints must have teeth.

Comparing with what has happened in other languages, which have stagnated or not stagnated at various rates, it is clear that what we need is a well-defined deprecation process.

The warnings should say something like: you had better upgrade this, otherwise it will stop working in the next version. Both maintainers and users should be aware of this threat.

I shouldn't worry. The next major version of ghc probably will not have base 3 any more. So the question is how we check and what warnings we give and to who. My current suggestion is to require upper bounds on the version of base and to warn when we end up selecting base 3 for older packages that do not specify an upper bound. Perhaps at some point we should start warning when the upper bound people specify for new uploads is < 4. What do you think the policy and mechanism should be? Duncan

Duncan Coutts wrote:

On Sun, 2009-02-01 at 15:56 +0100, Niklas Broberg wrote:

...

...

So in the next cabal-install release (which should be pretty soon now) configure will do the same thing and pick base 3 unless you specify build-depends base >= 4. ... and so there will never be any incentive for these many packages to migrate to base-4, which also has consequences for packages that do want to use base-4, but also want to depend on such packages.

Actually a package that uses base 4 can depend on other packages that use base 3. They all co-exist fine. This already happens now.

...

I would suggest as a less stagnating approach to issue a warning/hint when a package with no explicit version dependency for base fails to build.

So my plan is to make hackage require an upper bound on the version of base for all new packages. That should avoid the need to use the preferences hack the next time around.

Hrm. I can see why you might do that, if you keep the old base around. On the other hand, what if base 5 introduces less invasive changes? We have had, for awhile, a number of Haskell packages in Debian that require a version of GHC greater than version x and less than version x+1. This has not worked out entirely well for us. Granted, Cabal is different because GHC 6.10 has two versions of base. But still, I feel queasy about stating that my package won't work with base 5 when I haven't even seen base 5 yet. While we're at it, isn't this a more general problem that could occur elsewhere? Does base really need to be a special case? What if, say, utf8-string, binary, or some other commonly-used package had to break API?

As for what mechanisms we use to persuade package authors to use base 4 over base 3 for new releases, I'm open to suggestions.

Well, I'd say this will be difficult to achieve for at least two years, since GHC 6.8 is in current and future shipping versions of numerous Linux distributions, and such may well be the version of GHC most readily accessible to quite a few users. I am taking the approach of supporting *both*, but that is certainly not the most simple approach (it's not all that complex either, once you know how). -- John

On Sat, Jan 31, 2009 at 00:45, Krzysztof Skrzętnicki wrote:

On Sat, Jan 31, 2009 at 00:07, Valentyn Kamyshenko wrote:

...

Hello John,

I'm trying to install the new version of HDBC on my Mac (ghc version is 6.10.1), and get the following error:

# cabal upgrade --global Resolving dependencies... 'HDBC-2.0.0' is cached. Configuring HDBC-2.0.0... Preprocessing library HDBC-2.0.0... Preprocessing executables for HDBC-2.0.0... Building HDBC-2.0.0... [1 of 7] Compiling Database.HDBC.SqlValue ( Database/HDBC/SqlValue.hs, dist/build/Database/HDBC/SqlValue.o ) [2 of 7] Compiling Database.HDBC.ColTypes ( Database/HDBC/ColTypes.hs, dist/build/Database/HDBC/ColTypes.o ) [3 of 7] Compiling Database.HDBC.Statement ( Database/HDBC/Statement.hs, dist/build/Database/HDBC/Statement.o )

Database/HDBC/Statement.hs:113:9: Type constructor `Exception' used as a class In the instance declaration for `Exception SqlError' cabal: Error: some packages failed to install: HDBC-2.0.0 failed during the building phase. The exception was: exit: ExitFailure 1

I got the same error: Windows XP, GHC 6.10.1. For some reason imported Control.Exception module appears to be 6.8.* version.

= 4". Otherwise cabal-install selects base-3 which is the cause of the

I found a (temporary) fix: change dependancy in .cabal file to state "base problems. All best Christopher Skrzętnicki

On Sat, Jan 31, 2009 at 12:52:02AM +0100, Krzysztof Skrz??tnicki wrote:

...

I got the same error: Windows XP, GHC 6.10.1. For some reason imported Control.Exception module appears to be 6.8.* version.

...

= 4". Otherwise cabal-install selects base-3 which is the cause of the

I found a (temporary) fix: change dependancy in .cabal file to state "base problems.

Thanks for the hint for folks. Why would cabal-install select a different base than running Setup manually? I can't hard-code base >= 4 into .cabal because that would break for GHC 6.8 users. I have CPP code that selects what to compile based on GHC version. -- John

On Fri, Jan 30, 2009 at 06:40:29PM -0600, Austin Seipp wrote:

Excerpts from John Goerzen's message of Fri Jan 30 18:31:00 -0600 2009:

...

Why would cabal-install select a different base than running Setup manually?

I can't hard-code base >= 4 into .cabal because that would break for GHC 6.8 users. I have CPP code that selects what to compile based on GHC version.

-- John

I think it would be easiest to simply not use CPP and set a dependency on the extensible-exceptions package:

http://hackage.haskell.org/cgi-bin/hackage-scripts/package/extensible-except...

Using this package, you don't need CPP at all - just import 'Control.Exception.Extensible' instead of Control.Exception and you're done. This package works under both ghc 6.8 and 6.10 (when compiled under 6.10, it just re-exports all of Control.Exception, and on <= 6.10 it actually compiles the code) and means you don't have to have two awkward copies of the same code that use different exception-handling mechanisms.

Interesting. In its cabal file, I see this: if impl(ghc>=6.9) cpp-options: -DUSE_NEW_EXCEPTIONS build-depends: base>=4&&<5 else build-depends: base<4 interesting, that. Either: 1) Everyone will have the same problem or 2) That magic somehow disables the cabal-install voodoo that's messing everyone up. If it's #1, it doesn't help us. If it's #2, I could do the same thing in HDBC. Can anyone try to cabal-install extensible-exceptions on GHC 6.10 and let me know what happens? To be sure, I already had to use CPP for much of HDBC due to Hugs issues. I have a small bit of code in HDBC itself to deal with the interface differences, and then the backend drivers mostly don't have to care. They can call throwSqlError which uses throwDyn or throw as appropriate, and they can just be fine. It is probably not necessary to introduce another dep for the sake of saving as few lines of code as this. -- John

Niklas Broberg wrote:

...

Thanks for the hint for folks.

Why would cabal-install select a different base than running Setup manually?

Someone thought it was a good idea to make base-3 the preferred selection for cabal-install, to make the transition from base-3 to base-4 as smooth as possible. It may have worked for that, but it's causing a lot of problems now instead. Time for a new policy/version for cabal-install?

I've just uploaded HDBC 2.0.1 to cabal, that has the same hack as extensible-exceptions. Would one of you that had trouble with cabal-install on 2.0.0 please test and let me know if it's better? Thanks, -- John

The following change in .cabal file may solve the problems:--- flag base4 description: Choose base-4 if possible default: True library if flag(splitBase) if flag(base4) Build-Depends: base>=4, old-time, time, bytestring, containers, old-locale else Build-Depends: base>=3, old-time, time, bytestring, containers, old-locale else Build-Depends: base<3 Build-Depends: mtl, convertible >= 1.0.1, utf8-string --- It works for me and I *think* will also work for other setups, most notably GHC 6.8.*. All best Christopher Skrzętnicki On Sat, Jan 31, 2009 at 01:31, John Goerzen wrote:

On Sat, Jan 31, 2009 at 12:52:02AM +0100, Krzysztof Skrz??tnicki wrote:

...

...

I got the same error: Windows XP, GHC 6.10.1. For some reason imported Control.Exception module appears to be 6.8.* version.

...

= 4". Otherwise cabal-install selects base-3 which is the cause of the

I found a (temporary) fix: change dependancy in .cabal file to state "base problems.

Thanks for the hint for folks.

Why would cabal-install select a different base than running Setup manually?

I can't hard-code base >= 4 into .cabal because that would break for GHC 6.8 users. I have CPP code that selects what to compile based on GHC version.

-- John

On Sat, Jan 31, 2009 at 04:01, John Goerzen wrote:

On Sat, Jan 31, 2009 at 01:51:54AM +0100, Krzysztof Skrz??tnicki wrote:

...

The following change in .cabal file may solve the problems:--- flag base4 description: Choose base-4 if possible default: True

library if flag(splitBase) if flag(base4) Build-Depends: base>=4, old-time, time, bytestring, containers, old-locale else Build-Depends: base>=3, old-time, time, bytestring, containers, old-locale else Build-Depends: base<3 Build-Depends: mtl, convertible >= 1.0.1, utf8-string ---

It works for me and I *think* will also work for other setups, most notably GHC 6.8.*.

Not out of the box. The base4 flag will be on by default for GHC 6.8.x too, and they'll get the base >= 4 dep, which will break. They'd have to configure with the option to disable a flag (-f -base4 if memory serves), which is neither intuitive nor automatic.

No, not really. I downloaded GHC 6.8.3 to test it, and it works: D:\biblioteki\HDBC-2.0.0>cabal install -w c:\ghc\ghc-6.8.3\bin\ghc Resolving dependencies... Configuring HDBC-2.0.0.2... Preprocessing library HDBC-2.0.0.2... Preprocessing executables for HDBC-2.0.0.2... Building HDBC-2.0.0.2... [1 of 7] Compiling Database.HDBC.SqlValue ( Database/HDBC/SqlValue.hs, dist\build/Database/HDBC/SqlValue.o ) [2 of 7] Compiling Database.HDBC.ColTypes ( Database/HDBC/ColTypes.hs, dist\build/Database/HDBC/ColTypes.o ) [3 of 7] Compiling Database.HDBC.Statement ( Database/HDBC/Statement.hs, dist\build/Database/HDBC/Statement.o ) Database/HDBC/Statement.hs:15:0: Warning: Module `Control.Exception' is imported, but nothing from it is used, except perhaps instances visible in `Control.Exception' To suppress this warning, use: import Control.Exception() [4 of 7] Compiling Database.HDBC.Types ( Database/HDBC/Types.hs, dist\build/Database/HDBC/Types.o ) [5 of 7] Compiling Database.HDBC.Utils ( Database/HDBC/Utils.hs, dist\build/Database/HDBC/Utils.o ) [6 of 7] Compiling Database.HDBC.DriverUtils ( Database/HDBC/DriverUtils.hs, dist\build/Database/HDBC/DriverUtils.o ) [7 of 7] Compiling Database.HDBC ( Database/HDBC.hs, dist\build/Database/HDBC.o ) C:\ghc\ghc-6.10.1\bin\ar.exe: creating dist\build\libHSHDBC-2.0.0.2.a Installing library in C:\Program Files\Haskell\HDBC-2.0.0.2\ghc-6.8.3 Registering HDBC-2.0.0.2... Reading package info from "dist\\installed-pkg-config" ... done. Saving old package config file... done. Writing new package config file... done. Checking: C:\ghc\ghc-6.8.3\bin>ghc-pkg list HDBC C:/ghc/ghc-6.8.3\package.conf: HDBC-2.0.0.2 So both 6.8.3 and 6.10.1 work as expected. I can't remember the exact page in docs, but as far as i remember if Cabal fails to satisfy conditions with certain flag on its default settings the flag is changed and dependency calculation restarts. All best Christopher Skrzętnicki