Recompiling Hackage to estimate the impact of a change [Was: StricterLabelledFieldSyntax]

newer
ANN: Elerea version 1.x.x

older
ANN: haskell-src-exts-1.1.0

Johan Tibell

26 Jul 2009 26 Jul '09

2:21 a.m.

On Sun, Jul 26, 2009 at 3:34 AM, Ian Lynagh wrote:

...

Hi all,

I've made a ticket and proposal page for making the labelled field syntax stricter, e.g. making this illegal:

data A = A {x :: Int}

y :: Maybe A y = Just A {x = 5}

and requiring this instead:

data A = A {x :: Int}

y :: Maybe A y = Just (A {x = 5})

http://hackage.haskell.org/trac/haskell-prime/ticket/132

http://hackage.haskell.org/trac/haskell-prime/wiki/StricterLabelledFieldSynt...

In general, I think it would be a good idea to provide some statistics of how many packages would break as the result of a backwards incompatible change. Without that data I find it hard to do a cost-benefit analysis. So I hereby suggest that we make a recompile of Hackage a requirement for any breaking language changes. I understand that it might not be that easy to recompile all of Hackage at the moment so we should try to come up with some step-by-step instructions on how to do it. In the future it would be nice to have a little script that does it and spits out some statistics. Cheers, Johan

Attachments:

attachment.html (text/html — 1.6 KB)

Show replies by date

Don Stewart

26 Jul 26 Jul

2:25 a.m.

johan.tibell:

...

In general, I think it would be a good idea to provide some statistics of how many packages would break as the result of a backwards incompatible change. Without that data I find it hard to do a cost-benefit analysis. So I hereby suggest that we make a recompile of Hackage a requirement for any breaking language changes. I understand that it might not be that easy to recompile all of Hackage at the moment so we should try to come up with some step-by-step instructions on how to do it. In the future it would be nice to have a little script that does it and spits out some statistics.

Agreed. And it should be required as part of release processes for GHC. Duncan has described the process of building large amounts of Hackage, http://blog.well-typed.com/2009/03/regression-testing-with-hackage/ We used this prior to the ghc 6.10 release to find a few interesting type system bugs, and get in steps for what breaks and why, ultimately easing the upgrade process so that it affected only a couple of percent of packages. So, +1, let's take advantage of this centralization to improve overall quality standards. -- Don

Maurício

10:51 a.m.

New subject: Recompiling Hackage to estimate the impact of a change [Was: StricterLabelledFieldSyntax]

...

...
In general, I think it would be a good idea to provide some statistics of how many packages would break as the result of a backwards incompatible change.

...

Agreed. And it should be required as part of release processes for GHC.

One possible alternative, or complementary, action would be automating some kind of pool or voting from hackage packages maintainers. This would give a more informative answer than just breaks/doesn't break. The example that startted this thread would certainly break some packages, but I believe 99% of maintainers would say they could fix their code for compatibility before a new compiler release. (Even without voting, an automatic e-mail about language changes could be sent to maintainers, as many may even not beeing doing Haskell anymore. That could include some link to easily remove their e-mail from any package maintenance.) Maurício

Ian Lynagh

11:30 a.m.

On Sat, Jul 25, 2009 at 11:25:12PM -0700, Donald Bruce Stewart wrote:

...

johan.tibell:

...
In general, I think it would be a good idea to provide some statistics of how many packages would break as the result of a backwards incompatible change.

Agreed. And it should be required as part of release processes for GHC.

Duncan has described the process of building large amounts of Hackage,

http://blog.well-typed.com/2009/03/regression-testing-with-hackage/

That process requires too much effort. IIRC it took more than a day of actual human time to get the package list set up etc, and requires you to make some non-obvious decisions along the way. I think a better process would be more along the lines of: for p in $packages cabal install p rollback? except smarter, so that it doesn't try to install packages that it has already installed as a dependency of something else. Thanks Ian

Ian Lynagh

11:24 a.m.

On Sun, Jul 26, 2009 at 08:21:00AM +0200, Johan Tibell wrote:

...

On Sun, Jul 26, 2009 at 3:34 AM, Ian Lynagh wrote:

...
http://hackage.haskell.org/trac/haskell-prime/wiki/StricterLabelledFieldSynt...

In general, I think it would be a good idea to provide some statistics of how many packages would break as the result of a backwards incompatible change. Without that data I find it hard to do a cost-benefit analysis.

To some extent you are right, and if we had an easy to to get those stats when I would be in favour of doing so. But it is important to remember that a count of packages that break won't tell you how hard it would be to fix them. For example, the complete diff needed to fix old-time for StricterLabelledFieldSyntax was: - toClockTime cal{ctMonth=month', ctYear=year'} + toClockTime $ cal{ctMonth=month', ctYear=year'} Thanks Ian

Johan Tibell

3:48 p.m.

On Sun, Jul 26, 2009 at 5:24 PM, Ian Lynagh wrote:

...

On Sun, Jul 26, 2009 at 08:21:00AM +0200, Johan Tibell wrote:

...
On Sun, Jul 26, 2009 at 3:34 AM, Ian Lynagh wrote:

...
http://hackage.haskell.org/trac/haskell-prime/wiki/StricterLabelledFieldSynt...

...
In general, I think it would be a good idea to provide some statistics of how many packages would break as the result of a backwards incompatible change. Without that data I find it hard to do a cost-benefit analysis.

To some extent you are right, and if we had an easy to to get those stats when I would be in favour of doing so.

But it is important to remember that a count of packages that break won't tell you how hard it would be to fix them. For example, the complete diff needed to fix old-time for StricterLabelledFieldSyntax was:

- toClockTime cal{ctMonth=month', ctYear=year'} + toClockTime $ cal{ctMonth=month', ctYear=year'}

Right. So once you know what breaks you can investigate why and, as a part of the language change proposal, show how easy/hard it would be to fix breakages. I'm not arguing against breaking changes but for using the available data to make decisions. For example, when a redesign of haskell.org was brought up a while back the discussion could have greatly benefited from looking at web server logs to give valuable insight into user behavior on the site. -- Johan

5780

Age (days ago)

5780

Last active (days ago)

List overview

Download

5 comments

4 participants

participants (4)

Don Stewart
Ian Lynagh
Johan Tibell
Maurício