johan, how do you and Bryan have those jenkin's nodes setup? (I'm planning to setup something similar for my own use, and seeing how thats setup would be awesome) thanks -Carter On Thu, Feb 7, 2013 at 10:55 PM, Johan Tibell wrote:

On Thu, Feb 7, 2013 at 6:48 PM, Ben Lippmeier wrote:

...

Right now, the latest packages uploaded to Hackage get built with ghc-7.6 (only), and all the pages say "Built on ghc-7.6". By doing this we force *all* library developers to run GHC 7.6. I think this sends the clearest message about what the "real" GHC version is.

I don't know how closely users look at the "built on" line on the package page. Perhaps they look there, perhaps they don't.

Between Bryan and me we build all of our own packages and all HP packages on the latest 3 GHC versions:

http://ci.johantibell.com/ https://jenkins.serpentine.com/

That doesn't mean that it would be a bad idea to build all Hackage packages on using more GHC versions, but that won't happen until the Hackage project is unstuck (it has been stuck for years!).

-- Johan

-- You received this message because you are subscribed to the Google Groups "parallel-haskell" group. To unsubscribe from this group and stop receiving emails from it, send an email to parallel-haskell+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.

On 8 Feb 2013, at 05:18, Carter Schonwald wrote:

johan, how do you and Bryan have those jenkin's nodes setup?

(I'm planning to setup something similar for my own use, and seeing how thats setup would be awesome)

Likewise, I'm in the process of setting up Elastic Bamboo on EC2 for Cloud Haskell and would be very interested in seeing how you've dealt with multiple versions of GHC. Cheers, Tim

Hi Bryan, On 8 Feb 2013, at 11:53, Bryan O'Sullivan wrote:

On Fri, Feb 8, 2013 at 1:29 AM, Tim Watson wrote: Likewise, I'm in the process of setting up Elastic Bamboo on EC2 for Cloud Haskell and would be very interested in seeing how you've dealt with multiple versions of GHC.

It's easy to parameterize builds in Jenkins based on different values of an environment variable, so Johan and I just have different versions of GHC installed side by side, and then set $GHC_VERSION to "7.6 7.4 7.2 7.0 6.12" (or whatever), put /usr/local/$GHC_VERSION/bin at the front of $PATH, and the right thing happens.

Ok cool, that's pretty much what I had in mind but I wasn't sure about installing dependencies and using cabal-install. In my development environment I quickly found that installing multiple GHCs and haskell-platform releases got a bit messy, so I was wondering if there was a recognised 'best way' to do this. I'll probably replicate what I've done with other things (such as Erlang) and manage it with ${PREFIX}/ghc/versions/... and symlink ${PREFIX}/ghc/current/... to avoid the path switching. Hopefully telling cabal-install to use ${PREFIX}/ghc/current/lib will 'just work' when installing dependencies as I switch between ghc versions. Cheers! Tim

On Fri, Feb 8, 2013 at 2:28 PM, Simon Marlow wrote:

Does this strike a chord with anyone, or have I got the wrong impression and everyone is happy with the pace?

I am happy with the pace; I like the "release early, release often" approach. The HP is not forced to use the latest GHC, and the GHC download page already clearly directs users to the HP. Slowing down GHC releases will slow down adoption of new features, because while installing GHC might be "harder" than installing the HP, installing GHC HEAD is harder. My experience is that almost no one, apart from GHC devs, has HEAD, and they aren't willing to install it just to play with one new feature. Once a new compiler version is out, however, many people are happy to try it, even if it has no accompanying HP. Cheers, Pedro

Cheers, Simon

On 07/02/13 18:15, Simon Peyton-Jones wrote:

It’s fairly simple in my mind. There are two “channels” (if I understand Mark’s terminology right):****

** **

· Haskell Platform:****

o A stable development environment, lots of libraries known to work****

o Newcomers, and people who value stability, should use the Haskell Platform****

o HP comes with a particular version of GHC, probably not the hottest new one, but that doesn’t matter. It works.****

** **

· GHC home page downloads:****

o More features but not so stable****

o Libraries not guaranteed to work****

o Worth releasing, though, as a forcing function to fix bugs, and as a checkpoint for people to test, so that by the time the HP adopts a particular version it is reasonably solid.****

** **

So we already have the two channels Mark asks for, don’t we? One is called the Haskell Platform and one is called the GHC home page. ****

That leaves a PR issue: we really *don’t* want newcomers or Joe Users wanting the “new shiny”. They want the Haskell Platform, and as Mark says those users should not pay the slightest attention until it appears in the Haskell Platform.****

** **

So perhaps we principally need a way to point people away from GHC and towards HP? eg We could prominently say at every download point “Stop! Are you sure you want this? You might be better off with the Haskell Platform! Here’s why...”.****

** **

Have I understood aright? If so, how could we achieve the right social dynamics? ****

** **

Our goal is to let people who value stability get stability, while the hot-shots race along in a different channel and pay the price of flat tires etc.****

** **

PS: absolutely right to use 7.6.2 for the next HP. Don’t even think about 7.8.****

** **

Simon****

** **

** **

** **

*From:* Mark Lentczner [mailto:mark.lentczner@gmail.com]

*Sent:* 07 February 2013 17:43 *To:* Simon Peyton-Jones *Cc:* andreas.voellmy@gmail.com; Carter Schonwald; GHC users; Simon Marlow; parallel-haskell; kostirya@gmail.com; Edsko de Vries; ghc-devs@haskell.org *Subject:* Re: GHC 7.8 release?****

** **

I'd say the window for 7.8 in the platform is about closed. If 7.8 were to be release in the next two weeks that would be just about the least amount of time I'd want to see for libraries in the platform to get all stable with the GHC version. And we'd also be counting on the GHC team to be quickly responding to bugs so that there could be a point release of 7.8 mid-April. Historically, none of that seems likely.****

** **

So my current trajectory is to base HP 2013.2.0.0 on GHC 7.6.2.****

** **

Since 7.8 will seems like it will be released before May, we will be faced again with the bad public relations issue: Everyone will want the new shiny and be confused as to why the platform is such a laggard. We'll see four reactions:****

- New comers who are starting out and figure they should use the latest... Many will try to use 7.8, half the libraries on hackage won't work, things will be wonky, and they'll have a poor experience.**** - People doing production / project work will stay on 7.6 and ignore 7.8 for a few months.**** - The small group of people exploring the frontiers will know how to get things set up and be happy.**** - Eventually library authors will get around to making sure their stuff will work with it.****

I wish GHC would radically change it's release process. Things like 7.8 shouldn't be release as "7.8". That sounds major and stable. The web site will have 7.8 at the top. The warning to use the platform will fall flat because it makes the platform look out of date. Really, "7.8" should be in a different release channel, not on the front page. It should bake in that channel for six months - where only the third group of people will use it, until it is getting close to merge into main, at which point the fourth group will start to use it, so that the day it hits main, all the libraries just work. Ideally, the first two groups of people will not pay the slightest attention to it until it is further baked.****

** **

While we achievements of the GHC team are great, less than half of those 7.8 features seem interesting from the viewpoint of the aims of the platform. I don't think adding syntactic or type-level features are really all that important for Haskell at this juncture. And while I do like to see improvements in generated code and run-time performance, I think even those are less important than making crucial ecosystem improvements to things like package management, cross-compilation, and libraries.****

** **

- Mark****

-- You received this message because you are subscribed to the Google Groups "parallel-haskell" group. To unsubscribe from this group and stop receiving emails from it, send an email to parallel-haskell+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.

On Fri, Feb 08, 2013 at 02:28:20PM +0000, Simon Marlow wrote:

So I think, if anything, there's pressure to have fewer major releases of GHC. However, we're doing the opposite: 7.0 to 7.2 was 10 months, 7.2 to 7.4 was 6 months, 7.4 to 7.6 was 7 months. We're getting too efficient at making releases!

7.2 was billed as a "technology preview" rather than a regular stable release. However, it still required just as much effort as a regular stable release, both for us (we probably spent just as much time trying to make it bug-free, making builds, making docs, etc) and for the community (libraries still needed to adjust dependencies etc). One result of that extra effort was that the 7.4 release got delayed, and the delay was magnified by pushing it over the Christmas period. 7.6 was released roughly according to the regular yearly release plan (although the 7.4 delay made the gap between the two shorter). So in my opinion, 7.2 was a bad idea (but I don't think anyone knew that before we tried it), and I'd agree that we'd be better sticking to not-more-than-yearly major releases. I wouldn't oppose less-than-yearly (e.g. every 18 months) if that makes life easier for distros, library maintainers, the HP, etc. But I wouldn't advocate it either; from GHC's point of view, historically we've always had enough new stuff to justify a new major release after a year. Thanks Ian -- Ian Lynagh, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

On Fri, Feb 8, 2013 at 6:28 AM, Simon Marlow wrote:

For a while we've been doing one major release per year, and 1-2 minor releases. We have a big sign at the top of the download page directing people to the platform. We arrived here after various discussions in the past - there were always a group of people that wanted stability, and a roughly equally vocal group of people who wanted the latest bits. So we settled on one API-breaking change per year as a compromise.

Since then, the number of packages has ballooned, and there's a new factor in the equation: the cost to the ecosystem of an API-breaking release of GHC. All that updating of packages collectively costs the community a lot of time, for little benefit. Lots of package updates contributes to Cabal Hell. The package updates need to happen before the platform picks up the GHC release, so that when it goes into the platform, the packages are ready.

So I think, if anything, there's pressure to have fewer major releases of GHC. However, we're doing the opposite: 7.0 to 7.2 was 10 months, 7.2 to 7.4 was 6 months, 7.4 to 7.6 was 7 months. We're getting too efficient at making releases!

I think we want to decouple GHC "major" releases (as in, we did lots of work) from API breaking releases. For example, GCC has lots of major (or "big") releases, but rarely, if ever, break programs. I'd be delighted to see a release once in a while that made my programs faster/smaller/buggy without breaking any of them. -- Johan

+10^100 to Johan and Manuel. Breaking changes on pieces that aren't experimental is the main compatibility / new version pain, and I say this as someone who's spent time before and around the 7.4 and 7.6 releases testing out lots of major packages and sending a few patches to various maintainers. If there's a path to having a release strategy as Manuel suggests, and having an intermediate release with the new vector primops, type extensions and such goodness, then I'm all for it. A lot of these bits are things ill start using almost immediately in production / real software, esp if I'm not needing to patch every stable library beyond maybe relaxing versioning constraints. -Carter On Feb 8, 2013 9:05 PM, "Manuel M T Chakravarty" wrote:

I completely agree with Johan. The problem is to change core APIs too fast. Adding, say, SIMD instructions or having a new type extension (that needs to be explicitly activated with a -X option) shouldn't break packages.

I'm all for restricting major API changes to once a year, but why can't we have multiple updates to the code generator per year or generally release that don't affect a large number of packages on Hackage?

Manuel

Johan Tibell :

On Fri, Feb 8, 2013 at 6:28 AM, Simon Marlow wrote:

...

For a while we've been doing one major release per year, and 1-2 minor releases. We have a big sign at the top of the download page directing people to the platform. We arrived here after various discussions in the past - there were always a group of people that wanted stability, and a roughly equally vocal group of people who wanted the latest bits. So we settled on one API-breaking change per year as a compromise.

Since then, the number of packages has ballooned, and there's a new factor in the equation: the cost to the ecosystem of an API-breaking release of GHC. All that updating of packages collectively costs the community a lot of time, for little benefit. Lots of package updates contributes to Cabal Hell. The package updates need to happen before the platform picks up the GHC release, so that when it goes into the platform, the packages are ready.

So I think, if anything, there's pressure to have fewer major releases of GHC. However, we're doing the opposite: 7.0 to 7.2 was 10 months, 7.2 to 7.4 was 6 months, 7.4 to 7.6 was 7 months. We're getting too efficient at making releases!

I think we want to decouple GHC "major" releases (as in, we did lots of work) from API breaking releases. For example, GCC has lots of major (or "big") releases, but rarely, if ever, break programs.

I'd be delighted to see a release once in a while that made my programs faster/smaller/buggy without breaking any of them.

-- Johan

On Sat, Feb 9, 2013 at 6:27 AM, Simon Peyton-Jones wrote:

In short, I think we already have the situation that you desire. Perhaps we just need to market it better? ****

** **

Or am I mistaken?

Except the current question is about how ghc releases interact with the Platform; this thread was set off by a question about getting 7.6.2 into the next Platform.... And the main issue there is that ghc releases tend to break things and need a lot of testing in general to make it into the Platform; while this would be expected anyway, even a point release (7.6.2 vs. 7.6.1) of ghc tends to be moderately violent with respect to the Platform. Ideally, such a point release should not be difficult to slot in because it should be compatible modulo bug fixes, but with ghc's release strategy nobody has any confidence in it being that simple. -- brandon s allbery kf8nh sine nomine associates allbery.b@gmail.com ballbery@sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net

Simon Peyton-Jones :

If there's a path to having a release strategy as Manuel suggests, and having an intermediate release with the new vector primops, type extensions and such goodness, then I'm all for it. A lot of these bits are things ill start using almost immediately in production / real software, esp if I'm not needing to patch every stable library beyond maybe relaxing versioning constraints.

Let me suggest once more a possible path, along the lines you suggest · For people who value stability: use the Haskell Platform. Ignore GHC releases. · For people who want as many features as possible: use GHC releases. · For people who want to live on the bleeding edge: build HEAD from source

The Haskell Platform decides which GHC release to use, advertises that to package authors who do whatever updates are needed. HP may perfectly sensibly skip an entire release entirely.

In short, I think we already have the situation that you desire. Perhaps we just need to market it better?

Or am I mistaken?

There is one kink: for GHC releases to be *useful* substitutes for the HP for people who want medium stability, they must not change (expect maybe add to) the APIs in GHC versions that do not coincide with HP releases. Why? If they change APIs, many of the packages on Hackage will not build with these intermediate GHC releases, which makes them useless for anything, but testing GHC. Otherwise, I am perfectly happy with your suggestion. However, this is not the status quo. All (major) GHC releases do break critical packages on Hackage. Manuel

I agree too - I think it would be great to have non-API-breaking releases with new features. So let's think about how that could work. Some features add APIs, e.g. SIMD adds new primops. So we have to define non-API-breaking as a minor version bump in the PVP sense; that is, you can add to an API but not change it. As a straw man, let's suppose we want to do annual API releases in September, with intermediate non-API releases in February. Both would be classed as "major", and bump the GHC major version, but the Feb releases would only be allowed to bump minor versions of packages. (except perhaps the version of the GHC package, which is impossible to keep stable if we change the compiler). So how to manage the repos. We could have three branches, but that doesn't seem practical. Probably the best way forward is to develop new features on separate branches and merge them into master at the appropriate time - i.e. API-breaking feature branches could only be merged in after the Feb release. Thoughts? Cheers, Simon On 09/02/13 02:04, Manuel M T Chakravarty wrote:

I completely agree with Johan. The problem is to change core APIs too fast. Adding, say, SIMD instructions or having a new type extension (that needs to be explicitly activated with a -X option) shouldn't break packages.

I'm all for restricting major API changes to once a year, but why can't we have multiple updates to the code generator per year or generally release that don't affect a large number of packages on Hackage?

Manuel

Johan Tibell mailto:johan.tibell@gmail.com>:

...

On Fri, Feb 8, 2013 at 6:28 AM, Simon Marlow mailto:marlowsd@gmail.com> wrote:

For a while we've been doing one major release per year, and 1-2 minor releases. We have a big sign at the top of the download page directing people to the platform. We arrived here after various discussions in the past - there were always a group of people that wanted stability, and a roughly equally vocal group of people who wanted the latest bits. So we settled on one API-breaking change per year as a compromise.

Since then, the number of packages has ballooned, and there's a new factor in the equation: the cost to the ecosystem of an API-breaking release of GHC. All that updating of packages collectively costs the community a lot of time, for little benefit. Lots of package updates contributes to Cabal Hell. The package updates need to happen before the platform picks up the GHC release, so that when it goes into the platform, the packages are ready.

So I think, if anything, there's pressure to have fewer major releases of GHC. However, we're doing the opposite: 7.0 to 7.2 was 10 months, 7.2 to 7.4 was 6 months, 7.4 to 7.6 was 7 months. We're getting too efficient at making releases!

I think we want to decouple GHC "major" releases (as in, we did lots of work) from API breaking releases. For example, GCC has lots of major (or "big") releases, but rarely, if ever, break programs.

I'd be delighted to see a release once in a while that made my programs faster/smaller/buggy without breaking any of them.

-- Johan

Simon Marlow :

I agree too - I think it would be great to have non-API-breaking releases with new features. So let's think about how that could work.

Some features add APIs, e.g. SIMD adds new primops. So we have to define non-API-breaking as a minor version bump in the PVP sense; that is, you can add to an API but not change it.

As a straw man, let's suppose we want to do annual API releases in September, with intermediate non-API releases in February. Both would be classed as "major", and bump the GHC major version, but the Feb releases would only be allowed to bump minor versions of packages. (except perhaps the version of the GHC package, which is impossible to keep stable if we change the compiler).

So how to manage the repos. We could have three branches, but that doesn't seem practical. Probably the best way forward is to develop new features on separate branches and merge them into master at the appropriate time - i.e. API-breaking feature branches could only be merged in after the Feb release.

Thoughts?

That sounds sensible to me. Related to this, then, is the management of branches, which, I think, we can improve in two ways: (1) Make all library packages into submodules. (2) Fork-instead-of-branch and use GitHub pull requests. Re (1): submodules make tracking of synchronised branches across multiple repos simpler. Yes, they also have their pitfalls, but given that we are already using submodules extensively, we need to deal with those pitfalls anyway. So, why not reap the benefits, too? Re (2): we should encourage contributors to fork the GHC repos on GitHub and work in those. That makes it easy for everybody to build forks (which will be longer-lived under the above policy) and creating a fork doesn't require any special privileges in GHC repos. Finally, we can use GitHub pull requests to track contributions that are pending integration. This is IMHO also much nicer than attaching patches at Trac tickets. Manuel

On 09/02/13 02:04, Manuel M T Chakravarty wrote:

...

I completely agree with Johan. The problem is to change core APIs too fast. Adding, say, SIMD instructions or having a new type extension (that needs to be explicitly activated with a -X option) shouldn't break packages.

I'm all for restricting major API changes to once a year, but why can't we have multiple updates to the code generator per year or generally release that don't affect a large number of packages on Hackage?

Manuel

Johan Tibell mailto:johan.tibell@gmail.com>:

...

On Fri, Feb 8, 2013 at 6:28 AM, Simon Marlow mailto:marlowsd@gmail.com> wrote:

For a while we've been doing one major release per year, and 1-2 minor releases. We have a big sign at the top of the download page directing people to the platform. We arrived here after various discussions in the past - there were always a group of people that wanted stability, and a roughly equally vocal group of people who wanted the latest bits. So we settled on one API-breaking change per year as a compromise.

Since then, the number of packages has ballooned, and there's a new factor in the equation: the cost to the ecosystem of an API-breaking release of GHC. All that updating of packages collectively costs the community a lot of time, for little benefit. Lots of package updates contributes to Cabal Hell. The package updates need to happen before the platform picks up the GHC release, so that when it goes into the platform, the packages are ready.

So I think, if anything, there's pressure to have fewer major releases of GHC. However, we're doing the opposite: 7.0 to 7.2 was 10 months, 7.2 to 7.4 was 6 months, 7.4 to 7.6 was 7 months. We're getting too efficient at making releases!

I think we want to decouple GHC "major" releases (as in, we did lots of work) from API breaking releases. For example, GCC has lots of major (or "big") releases, but rarely, if ever, break programs.

I'd be delighted to see a release once in a while that made my programs faster/smaller/buggy without breaking any of them.

-- Johan

-- You received this message because you are subscribed to the Google Groups "parallel-haskell" group. To unsubscribe from this group and stop receiving emails from it, send an email to parallel-haskell+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.