Library Process (was Building "production stable" software in Haskell)
This discussion has sparked a question in my mind: What is the process for the inclusion of modules / packages in ghc, hugs and other compilers & interpreters? I thought the master plan was that less would come with the compiler / interpreter and the user would install packages using cabal. I was thus mildly surprised earlier this year to find e.g. data.bytestring was included with ghc. Now that I look at the haddock, it says this is a base package but doesn't say anything about its status. On the other hand, there are lots of modules that are base packages that are experimental. Should something experimental be a base package? And shouldn't all modules that are base packages declare their status? Perhaps these are questions for the libraries mailing list but I thought I'd start here. Yours confused of Kingston.
Hi
What is the process for the inclusion of modules / packages in ghc, hugs and other compilers & interpreters?
Propose to have the packaged added. There is a very low chance of this being accepted. The only packages to have recently been added were FilePath and ByteString, both of which were obvious deficiencies in the libraries. I'm now not aware of any hole that is likely to get plugged by bundling an additional library.
Should something experimental be a base package? And shouldn't all modules that are base packages declare their status?
No, they should all be reasonably stable. Things that are unstable are likely to move out of the standard libraries and just become hackage packages.
Perhaps these are questions for the libraries mailing list but I thought I'd start here.
Now there is a question I can't answer - I have no idea what should go down the libraries list and what should go down the haskell-cafe list. Perhaps someone could put up a guide of "if your question/comment is like this, send it on this list" Thanks Neil
Neil Mitchell wrote:
Hi
What is the process for the inclusion of modules / packages in ghc, hugs and other compilers & interpreters?
Propose to have the packaged added. There is a very low chance of this being accepted. The only packages to have recently been added were FilePath and ByteString, both of which were obvious deficiencies in the libraries. I'm now not aware of any hole that is likely to get plugged by bundling an additional library.
There still seem to be plenty of holes left. (No "standard" Finite Element Analysis or Digital Signal Processing libs for example.) Or put another way, what is so important about the holes that are filled by packages like GLUT,HGL,OpenGL,html,parsec,pretty,xhtml (to name a few) that require "standard" library implementations.
Should something experimental be a base package? And shouldn't all modules that are base packages declare their status?
No, they should all be reasonably stable. Things that are unstable are likely to move out of the standard libraries and just become hackage packages.
Libs are not "standard" simply because they happen to be bundled with ghc. And how is it that such unstable libs came to be bundled in the first place, given the alleged superior stability and quality control :-) Regards -- Adrian Hey
Dominic Steinitz
I thought the master plan was that less would come with the compiler / interpreter and the user would install packages using cabal.
Ideally, yes. I think a useful model would be GNU/Linux, where there is the Linux kernel, developed by core hackers, and then there are "distributions", which package up particular kernels with a load of different GNU libraries and utilities to form a complete operating environment. The maintainers of the distributions do not work on the kernel, but they do test their own combinations of kernel/libraries + utilities to ensure that everything plays together nicely. I would like to see the same separation forming between the ghc compiler itself (which would minimally include only the small number of libraries needed to build the compiler), and larger "distributions" which would be maintained by other people, and include much larger collections of packages that the maintainer has tested and verified to work together. In the best of all worlds, a Haskell "distribution" would include multiple compilers, not just ghc. Regards, Malcolm
On Tue, 2007-09-18 at 11:14 +0100, Malcolm Wallace wrote:
I would like to see the same separation forming between the ghc compiler itself (which would minimally include only the small number of libraries needed to build the compiler), and larger "distributions" which would be maintained by other people, and include much larger collections of packages that the maintainer has tested and verified to work together.
I think there is a niche for a subset of the hackage libraries providing an officially sanctioned standard library collection. Currently, hackage includes, well, everything. As such, it is a useful resource, but it would be useful to have a partitioning into two levels, where the "SLC" would only include libraries that meet specific criteria. Maybe: - considered stable - is portable - relies only on other standard libraries - avoids needless duplication of functionality - with a responsive, named maintainer (not libraries@) - with acceptable documentation and unit tests - required by at least one separate application -k
Hi
I think there is a niche for a subset of the hackage libraries providing an officially sanctioned standard library collection. Currently, hackage includes, well, everything. As such, it is a useful resource, but it would be useful to have a partitioning into two levels, where the "SLC" would only include libraries that meet specific criteria. Maybe:
- considered stable - is portable - relies only on other standard libraries - avoids needless duplication of functionality - with a responsive, named maintainer (not libraries@) - with acceptable documentation and unit tests - required by at least one separate application
I think there is a niche for this, but I don't think it should be an officially sanctioned collection - since otherwise everyone is just going to be debating how to add their library to this collection - and we are going to descend into voting and politics. Instead, I think several people should make their own personal list of "libraries they would vouch for" - which meet the criteria above AND they have personal positive experiences of. Off the top of my head my list would include gtk2hs, and that's about it. Thanks Neil
Instead, I think several people should make their own personal list of "libraries they would vouch for"
Ideally along with a cabal (or otherwise) install script that would set
everything up in one step.
"Neil Mitchell"
I think there is a niche for a subset of the hackage libraries providing an officially sanctioned standard library collection. Currently, hackage includes, well, everything. As such, it is a useful resource, but it would be useful to have a partitioning into two levels, where the "SLC" would only include libraries that meet specific criteria. Maybe:
- considered stable - is portable - relies only on other standard libraries - avoids needless duplication of functionality - with a responsive, named maintainer (not libraries@) - with acceptable documentation and unit tests - required by at least one separate application
I think there is a niche for this, but I don't think it should be an officially sanctioned collection - since otherwise everyone is just going to be debating how to add their library to this collection - and we are going to descend into voting and politics. Instead, I think several people should make their own personal list of "libraries they would vouch for" - which meet the criteria above AND they have personal positive experiences of. Off the top of my head my list would include gtk2hs, and that's about it. Thanks Neil _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe --- This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.
I thought the master plan was that less would come with the compiler / interpreter and the user would install packages using cabal.
Ideally, yes. I think a useful model would be GNU/Linux, where there is the Linux kernel, developed by core hackers, and then there are "distributions", which package up particular kernels with a load of different GNU libraries and utilities to form a complete operating environment. The maintainers of the distributions do not work on the kernel, but they do test their own combinations of kernel/libraries + utilities to ensure that everything plays together nicely.
I would like to see the same separation forming between the ghc compiler itself (which would minimally include only the small number of libraries needed to build the compiler), and larger "distributions" which would be maintained by other people, and include much larger collections of packages that the maintainer has tested and verified to work together. In the best of all worlds, a Haskell "distribution" would include multiple compilers, not just ghc.
i think that such separately maintained distributions would be one way to avoid the version skew chaos that might otherwise result from dropping all but the minimal set of libraries from ghc/hugs/.. distributions. now everybody take a step back to make room for the volunteers?-) a slightly more lightweight approach, that could grow into the above, would be if cabal packages supported nesting/grouping, so that sets of useful/proven libs could be bundled into a single package (the whole package builds if all the subpackages build individually, and form a coherent whole; that is, there'd need to be tests checking that the subpackages are actually compatible versions). then, instead of extra-libs in ghc/hugs/.., there'd be a standard-libs package on hackage showing exactly which haskell implementations it builds&tests with successfully and which not. and one could still get a whole consistent and maintained? bunch of standard libs in one go. </dream> claus
On Tuesday 18 September 2007 09:44, Dominic Steinitz wrote:
This discussion has sparked a question in my mind:
What is the process for the inclusion of modules / packages in ghc, hugs and other compilers & interpreters?
Personal interest of the people working on GHC et. al. ;-)
I thought the master plan was that less would come with the compiler / interpreter and the user would install packages using cabal. [...]
Although this statement might be a bit heretical on this list, I'll have to repeat myself again that Cabal, cabal-install, cabal-whatever will *never* be the right tool for the end user to install Haskell packages on platforms with their own packaging systems like RPM (the same holds for other systems, I just use RPM as an example here). This packaging system, and nothing else, will write into my /usr, otherwise chaos and versioning hell will threaten the system integrity. Cabal is a very fine tool to be used from RPM .spec files and to develop below one's own home directory, but it is not the tool of choice for the final installation. There are various reasons for this: * Imagine you want to find out to which package in which version a given file belongs. Impossible, if RPM is bypassed. * RPM offers functionality to verify the integrity of the installed SW, it can tell me which files are documentation, which ones are configuration files, etc. All this meta information has to be in a single system. * YaST, yum, etc. already have the notion of repositories, trust (via standard cryptographic methods) and resolving transitive dependencies, so we would re-implement things which are already there, well maintained and well-known to end users. * Imagine every language would force their end users to use specific tools for installation, this would be ridiculous. Personally I don't care at all about the details how Perl modules, some PHP/Python/... libraries etc. are installed on my system. This knowledge belongs to the packager who builds a nice RPM, mentioning all dependencies, so my favourite RPM tool can transitively resolve, download and install everything, offering a nice GUI if I like. No need to remember how to do this for Perl/PHP/Python/etc. Regarding the pros and cons of small, separate packages: In general I agree that this is the right direction, and this is what other languages do as well. There are e.g. tons of Perl/PHP/Python/Ruby RPMs available for my system, each offering a specific library, while the RPMs containing the interpreters/compilers are rather small. But: IMHO we are not there yet, because we still have to remove quite a few rough edges until we can smoothly offer e.g. an RPM repository with lots of small library RPMs (Cabal versionitis, updating the Haddock documentation index, etc.). Therefore, I'll continue to offer only "Sumo-style" RPMs for GHC + boot-libs + extra-libs for now, but I hope I can change this in the future. Another point: Although this is hard to believe nowadays, ;-) people are not always online, so "simply" installing what is missing might not always be an option. "Great, I'd really need the foobar-2.1 package now, but I'm currently 10000 feet above the Atlantic ocean..." The right way to tackle this problem is using meta packages, basically referencing lots of bundled small packages. RPM offers such a feature, and I guess other systems, too. On a laptop, such a meta package leading to the installation of tons of libraries is the right approach, on a fixed workstation the story might be different. Cheers, S.
On Tue, Sep 18, 2007 at 07:24:08PM +0200, Sven Panne wrote:
Although this statement might be a bit heretical on this list, I'll have to repeat myself again that Cabal, cabal-install, cabal-whatever will *never* be the right tool for the end user to install Haskell packages on platforms with their own packaging systems like RPM
this is a valid point. personally i only install cabal packages as --user and most tool-specific package managers (cpan etc) tend to offer this as an option. cabal is still necessary. it fills the gap where most OS package platforms won't provide support. even on the most supported platform (.deb for debian and ubuntu i presume), you still likely only get about 20% of what is in hackage on your system. what about everything else? i would prefer to have cabal in place of "make install". the only plausible solution i can see is generated OS packages (i.e. hackage hosts .deb, .rpm, and bsd packages on its own). this is likely the only realistic approach, but also periodically creates breakage too, particularly if the OS one day creates its own blessed packages. i would be willing to look into auto-generating freebsd packages, might be a fun project.
Sven Panne wrote:
On Tuesday 18 September 2007 09:44, Dominic Steinitz wrote:
This discussion has sparked a question in my mind:
What is the process for the inclusion of modules / packages in ghc, hugs and other compilers & interpreters?
Personal interest of the people working on GHC et. al. ;-)
I thought the master plan was that less would come with the compiler / interpreter and the user would install packages using cabal. [...]
Although this statement might be a bit heretical on this list, I'll have to repeat myself again that Cabal, cabal-install, cabal-whatever will *never* be the right tool for the end user to install Haskell packages on platforms with their own packaging systems like RPM (the same holds for other systems, I just use RPM as an example here).
I think you're identifying a non-problem here. Cabal was never intended to be used instead of the system's packaging tools for installing packages globally on the system, if you look back through the original Cabal design discussions you'll see this. We recognised the critical importance of working with, rather than around, emerge/ports/RPM/apt/whatever. Nowadays from a Cabal package you get make an RPM, a Windows installer, and the Gentoo folks have imported the entirety of Hackage. This is how it's meant to work. Cheers, Simon
Building Debian packages seems a bit hard to do for laymen like me.
Anyone have a script that takes a .cabal file, some additional meta
data and creates a .deb package? It would be really nice if we could
make this procedure easier. Every time I read some documentation on
how to create a Debian package, I feel like crying. ;)
Cheers,
Johan
On 9/19/07, Simon Marlow
Sven Panne wrote:
On Tuesday 18 September 2007 09:44, Dominic Steinitz wrote:
This discussion has sparked a question in my mind:
What is the process for the inclusion of modules / packages in ghc, hugs and other compilers & interpreters?
Personal interest of the people working on GHC et. al. ;-)
I thought the master plan was that less would come with the compiler / interpreter and the user would install packages using cabal. [...]
Although this statement might be a bit heretical on this list, I'll have to repeat myself again that Cabal, cabal-install, cabal-whatever will *never* be the right tool for the end user to install Haskell packages on platforms with their own packaging systems like RPM (the same holds for other systems, I just use RPM as an example here).
I think you're identifying a non-problem here. Cabal was never intended to be used instead of the system's packaging tools for installing packages globally on the system, if you look back through the original Cabal design discussions you'll see this. We recognised the critical importance of working with, rather than around, emerge/ports/RPM/apt/whatever.
Nowadays from a Cabal package you get make an RPM, a Windows installer, and the Gentoo folks have imported the entirety of Hackage. This is how it's meant to work.
Cheers, Simon _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
On Wed, Sep 19, 2007 at 11:05:05AM +0200, Johan Tibell wrote:
Building Debian packages seems a bit hard to do for laymen like me. Anyone have a script that takes a .cabal file, some additional meta data and creates a .deb package? It would be really nice if we could make this procedure easier. Every time I read some documentation on how to create a Debian package, I feel like crying. ;)
http://urchin.earth.li/pipermail/debian-haskell/2007-June/000322.html Thanks Ian
On 9/18/07, Sven Panne
Although this statement might be a bit heretical on this list, I'll have to repeat myself again that Cabal, cabal-install, cabal-whatever will *never* be the right tool for the end user to install Haskell packages on platforms with their own packaging systems like RPM (the same holds for other systems, I just use RPM as an example here).
Does RPM, etc., deal with the fact that Haskell library installations
are specific to a particular platform?
If I install GHC and some library, and later install Hugs or a later
GHC, how easy is it to make sure the library is available to the new
installations?
--
Dave Menendez
On Thursday 20 September 2007 16:33, David Menendez wrote:
Does RPM, etc., deal with the fact that Haskell library installations are specific to a particular platform?
It depends what you mean with "deal": If it is only making sure that a given binary library RPM matches the installed Haskell system, yes.
If I install GHC and some library, and later install Hugs or a later GHC, how easy is it to make sure the library is available to the new installations?
This is asking for much more, and the general answer for all mature packaging systems I know is: This will never be the case for *binary* packages. The simple reason is that for rebuilding packages (this is basically what you are asking for), you have different (and typically *many*) more dependencies. This is why e.g. RPM differentiates between "Requires" (i.e. normal runtime dependecies) and "Build-Requires" (dependencies for building an RPM). Although this situation is far from perfect, it is a direct consequence of the fact that there is no such thing as a "standard Haskell ABI" which is shared between all implementations and versions of them. The situation in C is much better (because it is much easier there), but even C++ suffered from ABI problems for several years on most platforms. Cheers, S.
On 9/23/07, Sven Panne
On Thursday 20 September 2007 16:33, David Menendez wrote:
Does RPM, etc., deal with the fact that Haskell library installations are specific to a particular platform?
It depends what you mean with "deal": If it is only making sure that a given binary library RPM matches the installed Haskell system, yes.
Here's what I mean:
Let's say I have more than one Haskell implementation on my computer,
e.g. GHC 6.6, GHC 6.7, and Hugs. (In MacPorts, these are the ghc,
ghc-devel, and hugs packages, respectively.)
Let's further say that I want to install the Edison API package.
Obviously, to get make it available in all three environments, the
library needs to be compiled twice and installed three times.
Using Cabal directly, I can simply run the configure/build/install
process three times with different configuration options.
Is this possible with systems like RPM/apt/port/etc?
--
Dave Menendez
Hello David, Sunday, September 23, 2007, 10:28:41 PM, you wrote:
Let's say I have more than one Haskell implementation on my computer, e.g. GHC 6.6, GHC 6.7, and Hugs. (In MacPorts, these are the ghc, ghc-devel, and hugs packages, respectively.)
Let's further say that I want to install the Edison API package. Obviously, to get make it available in all three environments, the library needs to be compiled twice and installed three times.
:) it should be compiled three times, too. different ghc versions has been never intended to be binary-compatible -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com
Bulat Ziganshin wrote:
Hello David,
Sunday, September 23, 2007, 10:28:41 PM, you wrote:
Let's say I have more than one Haskell implementation on my computer, e.g. GHC 6.6, GHC 6.7, and Hugs. (In MacPorts, these are the ghc, ghc-devel, and hugs packages, respectively.)
Let's further say that I want to install the Edison API package. Obviously, to get make it available in all three environments, the library needs to be compiled twice and installed three times.
:) it should be compiled three times, too. different ghc versions has been never intended to be binary-compatible
I thought so at first, too... after a minute I realized it probably means that Hugs doesn't _compile_ anything in advance of installation. So I guess the number is right at least, depending what "compile" is taken to mean. Isaac
On 9/23/07, Isaac Dupree
Bulat Ziganshin wrote:
Hello David,
Sunday, September 23, 2007, 10:28:41 PM, you wrote:
Let's say I have more than one Haskell implementation on my computer, e.g. GHC 6.6, GHC 6.7, and Hugs. (In MacPorts, these are the ghc, ghc-devel, and hugs packages, respectively.)
Let's further say that I want to install the Edison API package. Obviously, to get make it available in all three environments, the library needs to be compiled twice and installed three times.
:) it should be compiled three times, too. different ghc versions has been never intended to be binary-compatible
I thought so at first, too... after a minute I realized it probably means that Hugs doesn't _compile_ anything in advance of installation. So I guess the number is right at least, depending what "compile" is taken to mean.
I meant compilation in the sense of producing object code. Of course,
you still have to build the code for hugs (preprocessing and such), so
I guess the fact that you aren't actually compiling the code is
irrelevant.
My point was that I'm not aware of any packaging systems that don't
have a global "installed"/"not installed" bit for each package, which
isn't suited to handling Haskell libraries.
--
Dave Menendez
On Sun, 2007-09-23 at 21:17 -0400, David Menendez wrote:
My point was that I'm not aware of any packaging systems that don't have a global "installed"/"not installed" bit for each package, which isn't suited to handling Haskell libraries.
I don't agree - you are assuming there is a one to one correspondence between Cabal libraries and binary packages. The normal way to resolve this is that each source library gets compiled to multiple binary packages, which optionally can be unified as virtual packages 'providing' specific functionality. E.g. on my system (Ubuntu), I installed the virtual package 'ghc' to get GHC 6.6(.1?) which is actually provided by the package called 'ghc6'. In addition, I have the QuickCheck library installed, from a package named 'libghc6-quickcheck-dev', making room for libhugs-quickcheck-dev' etc. -k
David Menendez wrote:
Using Cabal directly, I can simply run the configure/build/install process three times with different configuration options.
Is this possible with systems like RPM/apt/port/etc?
Yes. In the case of RPM and dpkg, we prefix a library's name with the
name and version of the compiler name against which it's built. This
means that you can have hugs20051030-binary, ghc661-binary, and so on
all installed at once without a problem.
participants (17)
-
Adrian Hey -
brad clawsie -
Bryan O'Sullivan -
Bulat Ziganshin -
Claus Reinke -
David Menendez -
David Menendez -
Dominic Steinitz -
Ian Lynagh -
Isaac Dupree -
Johan Tibell -
Ketil Malde -
Malcolm Wallace -
Neil Mitchell -
Simon Marlow -
Sven Panne -
Thomas Hartman