Stepping back: what do we want from platform releases?

All, On reflection, we (Don and I) think we've asked a question that has too many assumptions behind it that are not universally agreed. Perhaps we would do better to look at the slightly more general issue. The question we asked about API stability in minor releases already assumes a lot about the frequency of major and minor releases and indeed what a major or minor release actually is; what their purpose is. We've actually started out backwards, we've assumed major and minor timetables and a version number scheme and then asked everyone what should fit that. Instead we should agree what the purpose of Haskell Platform releases are. Once we have decided that, then working out how frequent releases should be and the precise rules and version number schemes will be details that will be much easier to agree. So, what is the purpose of platform releases? Remember that we already have a system whereby any maintainer can make releases of their packages at any time and end users can download and install those releases. So what is the additional purpose of platform releases? * Bundling everything into nice easy end-user installers. This is certainly one purpose. For achieving just this we do not need anything more than a working snapshot of packages from hackage. * Testing. Making sure a set of packages work together. * Getting new features into the hands of users. Not everyone wants to download the latest and greatest from hackage. They want to use things tested together but still want new stuff occasionally. These points we think we all agree on, but none of them really tell us much about how frequent releases should be, as long as they're not so far apart that it defeats the third point about getting new features into the hands of users. * Telling distributions what they should package and telling users what they should have to be able to run the majority of relatively recent Haskell code. These are two purposes but closely related. * Telling developers what set of libs to test against to maximise their potential audience. These are about trying to make sure that users and developers are using the "same" stuff so that people can write programs and know that other people can run them. These require that the releases are the "same" between various operating systems (which the first 3 purposes to not strictly require) but more so that they should be the same between many users and many developers over some non-trivial period of time. This is the stability argument. If we agree with these purposes then we have to consider the balance between stability and progress in terms of new packages, new features and sometimes changing some things in incompatible ways. Note that these points do imply that new features (even compatible ones) cannot be added too frequently. Otherwise developers will write programs using those new features and users will not be able to run them. We would be asking users to upgrade too frequently. One strategy for balancing stability and change is to compress changes into specific points -- major releases -- rather than trickling them out every month. This is in contrast to Hackage which allows for very rapid change and feedback. * Synchronise cross-package changes. Reduce chaotic transitions, eg switching old-time -> time, or new exceptions. This is more ambitious. This implies some community process to decide on these kinds of cross-package changes and buy-in from package maintainers to follow through on the community decision in time for a particular platform release. -------- So we must consider what we are asking users, distributors, developers and maintainers to do. Remember also that we are not preventing developers from releasing new features in their Hackage releases and that the platform gives end users the tools necessary to get those latest and greatest releases. So in comparison to other systems we actually have a much better release valve for the pressure to get new features to users quickly. Duncan and Don

Duncan Coutts
All,
On reflection, we (Don and I) think we've asked a question that has too many assumptions behind it that are not universally agreed. Perhaps we would do better to look at the slightly more general issue.
The question we asked about API stability in minor releases already assumes a lot about the frequency of major and minor releases and indeed what a major or minor release actually is; what their purpose is.
My two cents:
My understanding of the purpose behind the Haskell Platform project is
that it:
* provides a convenient installer
* provides a batteries-included set of "canonical" libraries, with a
lukewarm guarantee that the libraries work and have been tested
together.
The concrete problems that we're trying to solve:
* bootstrapping a functioning Haskell system (with cabal & a
reasonable library set) is a multi-step, time-consuming process
* hackage (thankfully) has a low barrier to entry, but that also means
there's a mishmash of stuff there in varying states of
quality/testedness/"only-works-with-ghc-6.6"-ism. Evaluating the
quality of a library can be difficult for non-experts.
* cabal is pretty poor at handling dependency conflicts. I've wiped
out & reinstalled the entire haskell stack more times than I would
have liked over hard-to-resolve dependency conflicts, and I can't be
alone in that. A goal of HP is to ameliorate that.
If we're talking about the definitions of major vs minor releases, I'm
in the camp that says that if you depend on a major version of the
haskell platform, your code should recompile cleanly against minor
releases on that line (i.e. no backwards-incompatible changes). For a
minor release, bugfixes are clearly okay, new features or modules are
probably okay (if limited in scope), and API changes are a no-no. I
suspect this meshes with most people's common-sense understanding.
***
Something that isn't clear to me is: how are we going to encourage
people posting libs to Hackage to target the platform? As a library
author, it would be really nice to be able to say:
Build-Depends: haskell-platform >= 2009.2 && < 2009.3
and then be able to "import Network". Instead you get:
Foo.hs:3:7:
Could not find module `Network':
it is a member of package network-2.2.1.1, which is hidden
If you want to target the platform as a library author, then you have to
go through your long laundry-list of dependencies and assign an exact
version to each, and you have to repeat this process every time a new
version of the platform comes out. I reckon that for many people, the
cost/benefit analysis is going to come out to "I can't be bothered."
Can this be fixed in cabal? Should we export a module "Haskell.Platform"
that re-exports all of the platform modules? (That's a modest proposal,
of course, compilation time would go through the roof, but it'd be
convenient, and unless I'm missing something "convenient" is the raison
d'etre here.)
G.
--
Gregory Collins

On Sun, May 10, 2009 at 12:34 PM, Duncan Coutts wrote: So
what is the additional purpose of platform releases? * Getting new features into the hands of users. Not everyone wants
to download the latest and greatest from hackage. They want to
use things tested together but still want new stuff
occasionally. This tangentially captures something that is important, but which I think
deserves a much more prominent mention as an explicit goal: providing a
"blessed" set of libraries that is large enough to be useful for many common
programming tasks. In fact, I'd even suggest that the goal not just be to
round out the collection of useful libraries, but that they be "good"
according to criteria of efficiency, quality, and documentation, too. For
instance, the Platform is currently missing a library for XML processing,
but the obvious candidates on Hackage don't (in my eyes, at least) meet most
of my criteria of "good".
It might be that identifying those gaps in "well roundedness" would be a
helpful step towards encouraging the more focused development of suitable
libraries, which I think would be a very good piece of fall-out from the
Platform process.

bos:
On Sun, May 10, 2009 at 12:34 PM, Duncan Coutts
wrote: So what is the additional purpose of platform releases?
* Getting new features into the hands of users. Not everyone wants to download the latest and greatest from hackage. They want to use things tested together but still want new stuff occasionally.
This tangentially captures something that is important, but which I think deserves a much more prominent mention as an explicit goal: providing a "blessed" set of libraries that is large enough to be useful for many common programming tasks. In fact, I'd even suggest that the goal not just be to round out the collection of useful libraries, but that they be "good" according to criteria of efficiency, quality, and documentation, too. For instance, the Platform is currently missing a library for XML processing, but the obvious candidates on Hackage don't (in my eyes, at least) meet most of my criteria of "good".
It might be that identifying those gaps in "well roundedness" would be a helpful step towards encouraging the more focused development of suitable libraries, which I think would be a very good piece of fall-out from the Platform process.
Great point. I've captured it here: http://trac.haskell.org/haskell-platform/wiki/AddingPackages

When I first began using Haskell about 2 - 3 years ago, a major
frustration for me was the number of "bit-rotted" libraries I found. I
am also a Windows user which compounded the problem, since many
libraries were not Windows friendly at the time. Finally, because I
was learning Haskell, I did not have the wherewithal to distinguish
trivial versioning errors from real problems.
Things are much better now with Hackage and cabal, but even so the
platform can provide ease-of-use and stability for those wanting to
use Haskell, but not wanting to build everything themselves. Bit-rot
can't be prevented, but the platform can help make it obvious to a
given user if a older library can still be used.
The School of Expression Graphics package is a perfect example of how
the platform could help. That package never bit-rotted but it has
changed significantly from the book and getting it working is
non-trivial. The package is also most likely to be used by newbies,
which makes using the library that much harder and off-putting. If
the SOE package was tied to a particular platform release, then it
would be a lot easier to get it working, keep it working, and explain
to others how to use it.
I really like the idea of the platform and encourage the work on it to continue!
On Sun, May 10, 2009 at 12:34 PM, Duncan Coutts
All,
On reflection, we (Don and I) think we've asked a question that has too many assumptions behind it that are not universally agreed. Perhaps we would do better to look at the slightly more general issue.
The question we asked about API stability in minor releases already assumes a lot about the frequency of major and minor releases and indeed what a major or minor release actually is; what their purpose is.
We've actually started out backwards, we've assumed major and minor timetables and a version number scheme and then asked everyone what should fit that. Instead we should agree what the purpose of Haskell Platform releases are. Once we have decided that, then working out how frequent releases should be and the precise rules and version number schemes will be details that will be much easier to agree.
So, what is the purpose of platform releases? Remember that we already have a system whereby any maintainer can make releases of their packages at any time and end users can download and install those releases. So what is the additional purpose of platform releases?
* Bundling everything into nice easy end-user installers. This is certainly one purpose. For achieving just this we do not need anything more than a working snapshot of packages from hackage.
* Testing. Making sure a set of packages work together.
* Getting new features into the hands of users. Not everyone wants to download the latest and greatest from hackage. They want to use things tested together but still want new stuff occasionally.
These points we think we all agree on, but none of them really tell us much about how frequent releases should be, as long as they're not so far apart that it defeats the third point about getting new features into the hands of users.
* Telling distributions what they should package and telling users what they should have to be able to run the majority of relatively recent Haskell code. These are two purposes but closely related.
* Telling developers what set of libs to test against to maximise their potential audience.
These are about trying to make sure that users and developers are using the "same" stuff so that people can write programs and know that other people can run them. These require that the releases are the "same" between various operating systems (which the first 3 purposes to not strictly require) but more so that they should be the same between many users and many developers over some non-trivial period of time. This is the stability argument.
If we agree with these purposes then we have to consider the balance between stability and progress in terms of new packages, new features and sometimes changing some things in incompatible ways.
Note that these points do imply that new features (even compatible ones) cannot be added too frequently. Otherwise developers will write programs using those new features and users will not be able to run them. We would be asking users to upgrade too frequently.
One strategy for balancing stability and change is to compress changes into specific points -- major releases -- rather than trickling them out every month. This is in contrast to Hackage which allows for very rapid change and feedback.
* Synchronise cross-package changes. Reduce chaotic transitions, eg switching old-time -> time, or new exceptions.
This is more ambitious. This implies some community process to decide on these kinds of cross-package changes and buy-in from package maintainers to follow through on the community decision in time for a particular platform release.
--------
So we must consider what we are asking users, distributors, developers and maintainers to do.
Remember also that we are not preventing developers from releasing new features in their Hackage releases and that the platform gives end users the tools necessary to get those latest and greatest releases. So in comparison to other systems we actually have a much better release valve for the pressure to get new features to users quickly.
Duncan and Don
_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

Duncan Coutts wrote:
So we must consider what we are asking users, distributors, developers and maintainers to do.
Remember also that we are not preventing developers from releasing new features in their Hackage releases and that the platform gives end users the tools necessary to get those latest and greatest releases. So in comparison to other systems we actually have a much better release valve for the pressure to get new features to users quickly.
Personally, I think the point of the HP should not include "get new features to users quickly", but rather should have the different focus of "get communally-accepted features to users quickly". As you say, Hackage covers the former quite well already. Haskell is an experimental language (and that's a good thing), but not all experiments are successful. Hackage provides an excellent playground for having these experiments in front of a large audience in order to tease out their feasibility. Since the platform is an attempt to collect the community consensus on what "good Haskell" is (in addition to the one-click-install goal), I think therefore that "new" is not the appropriate metric for feature inclusion. I don't think the HP committee should get bogged down in questions of deciding what the community considers good, but for the majority of cases the answers should be obvious. When the current Haskell community would obviously include something, then it should be included in the HP so that newcomers can easily install and become familiar with things the community assumes familiarity with. Similarly, when a current community member assumes familiarity with these basic tools, their code should be able to make the same assumptions. Developers have an intuition for which libraries are canon vs which are "real" dependencies, and it seems like the HP vs Hackage should capture the flavor of that distinction. -- Live well, ~wren
participants (6)
-
Bryan O'Sullivan
-
Don Stewart
-
Duncan Coutts
-
Gregory Collins
-
Justin Bailey
-
wren ng thornton