On 30 August 2012 15:34, Alexander Bernauer wrote:

Hi

I agree that automatic code migration can solve this issue in large parts. The Python folks have done this to mitigate the transition from version 2 to version 3 [1].

On Thu, Aug 30, 2012 at 03:03:05PM +0200, Alexander Kjeldaas wrote:

...

perl -ni -e 'print unless /import Prelude hiding \(catch\)/' $(git grep 'import Prelude hiding (catch)')

I don't think regular expressions are powerful enough. This example does not match on hiding multiple names, for instance.

This was just an example, the CInt regexp handles multiple names, so regexps have no problems handling this. And it's a simple "git grep 'import.*Prelude.*catch'" to see if this actually is a problem or not. My point is that this works, fixes 99% of the cases, and is 1000x less work overall. Alexander

But writing proper 'HsModule -> HsModule' functions should be doable.

And when each release comes with a bunch of such functions, packages could be automatically migrated.

Greetings

Alex

[1] http://docs.python.org/library/2to3.html

-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAlA/a/oACgkQevm6Dd/q44nJXQCffaxEJ/NZEftgoZ7viAWMuBO3 +jkAnRTw+VCMQn1k9NibyKpkGMtwvrQw =ds3M -----END PGP SIGNATURE-----

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Whoops, I messed up that first example: module PreludePre_7_6 (module Prelude, catch) where import Prelude import qualified System.IO.Error as E catch = E.catch On Thu, Aug 30, 2012 at 12:16 PM, Michael Sloan wrote:

A good way to specify such refactorings is as a Haskell module. For example:

module PreludePre_7_6 where

import Prelude hiding ( catch )

Or, an example of avoiding the Eq / Show / Num debacle:

module PreludePre_7_4 (module Prelude, Num) where

import Prelude hiding ( Num ) import qualified Prelude as P

instance template (Eq a, Show a) => Num a where instance P.Num a

(Uses my instance template proposal - though with different keywords - https://github.com/mgsloan/instance-templates )

Let's say we wanted to rename "catch" to "unsafeCatch"

module PreludePreUnsafeCatch (module Prelude, catch) where

import Prelude hiding (unsafeCatch)

catch = unsafeCatch

Then, an automated refactoring system is "simply" a sourcecode-based module inliner. Quite a bit of effort, certainly, but it's much better than coming up with a whole new notation for refactorings.

-Michael

On Thu, Aug 30, 2012 at 6:34 AM, Alexander Bernauer wrote:

...

Hi

I agree that automatic code migration can solve this issue in large parts. The Python folks have done this to mitigate the transition from version 2 to version 3 [1].

On Thu, Aug 30, 2012 at 03:03:05PM +0200, Alexander Kjeldaas wrote:

...

perl -ni -e 'print unless /import Prelude hiding \(catch\)/' $(git grep 'import Prelude hiding (catch)')

I don't think regular expressions are powerful enough. This example does not match on hiding multiple names, for instance.

But writing proper 'HsModule -> HsModule' functions should be doable.

And when each release comes with a bunch of such functions, packages could be automatically migrated.

Greetings

Alex

[1] http://docs.python.org/library/2to3.html

-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAlA/a/oACgkQevm6Dd/q44nJXQCffaxEJ/NZEftgoZ7viAWMuBO3 +jkAnRTw+VCMQn1k9NibyKpkGMtwvrQw =ds3M -----END PGP SIGNATURE-----

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

On 8/30/12 10:26 AM, Erik Hesselink wrote:

Note that this does not work if you want to support multiple versions of GHC, and might not work in general, since

* The hiding of catch is needed for preludes that still have it, since otherwise it will probably conflict with the one from Control.Exception. * Older versions do not have constructors for the FFI types, leading to warnings (or errors, I'm not sure).

In 6.12, they're warnings. And those warnings only show up with -Wall enabled. Though there does not appear to be an appropriate -fno-warn-foo to disable only these warnings while leaving the rest of -Wall intact. (If anyone knows of such a flag in 6.12, I'd be much obliged.)

* Packages might not work with the new bytestring version, since the API has breaking changes (although they're supposed to be minor).

For the first two, you need to add some CPP, for the last, you need actual testing.

Actually, that can be more problematic than you suspect. In particular, hsc2hs does not play nicely with Cabal's MIN_VERSION_foo(1,2,3) macros. So if you're using hsc2hs, which is common for certain FFI uses, then you have to put up with the warnings or rely on the __GLASGOW_HASKELL__ macro in lieu of the MIN_VERSION_foo(1,2,3) macros. It's doable, but it's uglier and more hackish than it ought to be. -- Live well, ~wren

On Thu, 30 Aug 2012, Alexander Kjeldaas wrote:

This is very unfortunate, but this is crucially a tooling issue. I am going to wave my hands, but..

Ignore the mapreduce in the following video, but look at the use of clang to do automatic refactoring of C++. This is *incredibly* powerful in dealing with updates to APIs.

http://www.llvm.org/devmtg/2011-11/videos/Carruth_ClangMapReduce-desktop.mp4

But without all that fancy tech, *just* having all of Hackage source code in one repository and using perl/regexps, fixing these types of issues is O(1) instead of O(n).

All of the issues you mention seems to be fixable by a few lines of perl *if we had the repository*.

Better to do this with sexps. ad repositories: Part of the general problem of managing a repository is close to the problem of inferring a good type for (the value of) an expression. The style of constraints is similar. Now the design problem is: 1. Arrange a general system for the specification of the constraints. 2. Design a systematic way of giving both advice and direct commands to the system. This subsystem would be used to set up the default policy. 3. Choose a constraint solver. Maybe worth looking at: http://en.wikipedia.org/wiki/Nix_package_manager [page was last modified on 17 July 2012 at 20:20] oo--JS.

[a few hours later]

Actually, I went and downloaded all of hackage, put it into a git repository and fixed these issues:

Fix catch perl -ni -e 'print unless /import Prelude hiding \(catch\)/' $(git grep 'import Prelude hiding (catch)')

Fix CInt constructors (lots of other stuff from Foreign.C.Types not fixed though) perl -p -i -e 's/^import Foreign.C.Types(.*)CInt([^(])/import Foreign.C.Types${1}CInt(..)${1}/g' $(git grep -l '^import.*CInt')

Fix bytestring versioning perl -p -i -e 's/bytestring( +)>=([0-9. &]+)<([ ]*)0.10/bytestring$1>=$2<${3}0.11/g' $(git grep 'bytestring.*< *0\.')

Patch to hackage: http://ge.tt/6Cb5ErM/v/0

I understand that this doesn't help anyone, but if there was a way fix, upload, and get *consensus* on a few regexps like this, then doing API changes wouldn't be such a headache.

Alexander

On 30 August 2012 07:26, Bryan O'Sullivan wrote:

...

Since the release of the GHC 7.6 RC, I've been going through my packages and fixing up build problems so that people who upgrade to 7.6 will have a smooth ride.

Sad to say, my experience of 7.6 is that it has felt like a particularly rough release for backwards incompatibility. I wanted to quantify the pain, so I did some research, and here's what I found.

I maintain 25 open source Haskell packages. Of these, the majority have needed updates due to the GHC 7.6 release:

- base16-bytestring - blaze-textual - bloomfilter - configurator - criterion - double-conversion - filemanip - HDBC-mysql - mwc-random - pcap - pool - riak-haskell-client - snappy - text - text-format - text-icu

That's 16 out of 25 packages I've had to update. I've also either reported bugs on, or had to fix, several other people's packages along the way (maybe four?). So let's say I've run into problems with 20 out of the combined 29 packages of mine and my upstreams.

The reasons for these problems fall into three bins:

- Prelude no longer exports catch, so a lot of "import Prelude hiding (catch)" had to change. - The FFI now requires constructors to be visible, so "CInt" has to be imported as "CInt(..)". - bytestring finally got bumped to 0.10, so many upper bounds had to be relaxed (*cf* my suggestion that the upper-bounds-by-default policy is destructive).

It has been a lot of work to test 29 packages, and then modify, rebuild, and release 20 of them. It has consumed most of my limited free time for almost two weeks. Worse, this has felt like make-work, of no practical benefit to anyone beyond scrambling to restore the status quo ante.

If over half of my packages needed fixing, I'm alarmed at the thought of the effects on the rest of Hackage.

I'm torn over this. I understand and agree with the impetus to improve the platform by tidying things up, and yet just two seemingly innocuous changes (catch and FFI) have forced me to do a bunch of running to stand still.

I don't have any suggestions about what to do; I know that it's hard to estimate the downstream effects of what look like small changes. And so I'm not exactly complaining. Call this an unhappy data point.

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

On Fri, 31 Aug 2012, Alexander Kjeldaas wrote:

I think you're making this way harder than it really is.

What 99% of people need is that hackage packages builds with the latest haskell platform, and/or with bleeding edge ghc, and with the latest versions of its dependencies.

Thus for every dependency there is only one possible version - the latest one, and there are only a couple of compilers.

Having "special interest groups" for ghc 6.12 support and old versions of text is fine, but I think it is a pretty uninteresting problem to solve.

Likewise, supporting/fixing packages where the author for some reason *requires* use of a non-current version of some other package is also quite uninteresting (or at least outside the scope of my needs). Such a package is basically just a relic.

Alexander

If that is the set of constraints you must meet, namely that for every library you wish to use, the same platform is specified as the only platform you want, yes, I agree. I am also sympathetic to imposing such tight management of the repository. (This tight management could consist of just well checked annotations as to what works with what.) But there are other cases, for example, testing various combinations of libraries for speed and memory use. And such a flexible tool will be of use when the Move To One Platform comes. Being able to pick up code and modify code, and to write code to do this, is part of the tradition today called "functional programming". The Cut Over would be done with a Big Haskell Program that would test/re-write/test etc. until every line of code was updated and a database of interactions produced and made available in convenient form. oo--JS.

On 30 August 2012 22:26, Jay Sulzberger wrote:

...

On Thu, 30 Aug 2012, Alexander Kjeldaas wrote:

This is very unfortunate, but this is crucially a tooling issue. I am

...

going to wave my hands, but..

Ignore the mapreduce in the following video, but look at the use of clang to do automatic refactoring of C++. This is *incredibly* powerful in dealing with updates to APIs.

http://www.llvm.org/devmtg/**2011-11/videos/Carruth_** ClangMapReduce-desktop.mp4http://www.llvm.org/devmtg/2011-11/videos/Carruth_ClangMapReduce-desktop.mp4

But without all that fancy tech, *just* having all of Hackage source code in one repository and using perl/regexps, fixing these types of issues is O(1) instead of O(n).

All of the issues you mention seems to be fixable by a few lines of perl *if we had the repository*.

Better to do this with sexps.

ad repositories: Part of the general problem of managing a repository is close to the problem of inferring a good type for (the value of) an expression. The style of constraints is similar. Now the design problem is:

1. Arrange a general system for the specification of the constraints.

2. Design a systematic way of giving both advice and direct commands to the system. This subsystem would be used to set up the default policy.

3. Choose a constraint solver.

Maybe worth looking at:

http://en.wikipedia.org/wiki/**Nix_package_managerhttp://en.wikipedia.org/wiki/Nix_package_manager [page was last modified on 17 July 2012 at 20:20]

oo--JS.

...

[a few hours later]

Actually, I went and downloaded all of hackage, put it into a git repository and fixed these issues:

Fix catch perl -ni -e 'print unless /import Prelude hiding \(catch\)/' $(git grep 'import Prelude hiding (catch)')

Fix CInt constructors (lots of other stuff from Foreign.C.Types not fixed though) perl -p -i -e 's/^import Foreign.C.Types(.*)CInt([^(])/**import Foreign.C.Types${1}CInt(..)${**1}/g' $(git grep -l '^import.*CInt')

Fix bytestring versioning perl -p -i -e 's/bytestring( +)>=([0-9. &]+)<([ ]*)0.10/bytestring$1>=$2<${3}**0.11/g' $(git grep 'bytestring.*< *0\.')

Patch to hackage: http://ge.tt/6Cb5ErM/v/0

I understand that this doesn't help anyone, but if there was a way fix, upload, and get *consensus* on a few regexps like this, then doing API changes wouldn't be such a headache.

Alexander

On 30 August 2012 07:26, Bryan O'Sullivan wrote:

Since the release of the GHC 7.6 RC, I've been going through my packages

...

and fixing up build problems so that people who upgrade to 7.6 will have a smooth ride.

Sad to say, my experience of 7.6 is that it has felt like a particularly rough release for backwards incompatibility. I wanted to quantify the pain, so I did some research, and here's what I found.

I maintain 25 open source Haskell packages. Of these, the majority have needed updates due to the GHC 7.6 release:

- base16-bytestring - blaze-textual - bloomfilter - configurator - criterion - double-conversion - filemanip - HDBC-mysql - mwc-random - pcap - pool - riak-haskell-client - snappy - text - text-format - text-icu

That's 16 out of 25 packages I've had to update. I've also either reported bugs on, or had to fix, several other people's packages along the way (maybe four?). So let's say I've run into problems with 20 out of the combined 29 packages of mine and my upstreams.

The reasons for these problems fall into three bins:

- Prelude no longer exports catch, so a lot of "import Prelude hiding (catch)" had to change. - The FFI now requires constructors to be visible, so "CInt" has to be imported as "CInt(..)". - bytestring finally got bumped to 0.10, so many upper bounds had to be relaxed (*cf* my suggestion that the upper-bounds-by-default policy

is destructive).

It has been a lot of work to test 29 packages, and then modify, rebuild, and release 20 of them. It has consumed most of my limited free time for almost two weeks. Worse, this has felt like make-work, of no practical benefit to anyone beyond scrambling to restore the status quo ante.

If over half of my packages needed fixing, I'm alarmed at the thought of the effects on the rest of Hackage.

I'm torn over this. I understand and agree with the impetus to improve the platform by tidying things up, and yet just two seemingly innocuous changes (catch and FFI) have forced me to do a bunch of running to stand still.

I don't have any suggestions about what to do; I know that it's hard to estimate the downstream effects of what look like small changes. And so I'm not exactly complaining. Call this an unhappy data point.

______________________________**_________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/**mailman/listinfo/haskell-cafehttp://www.haskell.org/mailman/listinfo/haskell-cafe

______________________________**_________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/**mailman/listinfo/haskell-cafehttp://www.haskell.org/mailman/listinfo/haskell-cafe