Here is what I propose to do regarding the libraries we ship with GHC 6.6. Comments are appreciated, since we need to finalise this story before the release candidate at the end of this week. For a while we have had the goal of devolving as many of the libraries that ship with GHC as possible. With Cabal, this is now possible: you can download GHC and then install 3rd-party packages with little hassle, or upgrade libraries that came with your GHC installation. So the days of the monolithic GHC+libraries installations should perhaps come to an end. On the other hand, it's often quite useful to be able to install a single distribution and get a comprehensive set of libraries. The folks who package GHC for the deeply package-based OSs like Debian, Gentoo and *BSD would like GHC to come with a minimal set of libraries and install the others as separate packages. Another consideration is that it's useful for our nightly builds to build as many libraries as possible, so we can help catch bitrot. However, the default GHC build now takes a very long time mostly due to the huge set of libraries, and most people building GHC don't need to build eg. OpenGL by default. What's more, the maintainers of many of these libraries would like to be able to make releases independently of GHC. So here's what I propose: - A "GHC source tree" will contain a core set of packages. This is what you will get if you download ghc-6.6-src.tar.bz2, or do "sh darcs-all get" in a darcs repo. The core packages are: base, haskell98, template-haskell, readline, stm, Cabal, unix, Win32 plus hopefully regex-base and regex-posix if I integrate them this week. These are the packages required to bootstrap GHC, or those with deep GHC dependencies (template-haskell & stm). - The core packages will have a stable API across patchlevel releases of GHC (perhaps with the exception of Cabal). We will fork the darcs repos of the core packages where necessary. - A source tree can optionally be populated with more packages, which will be included in the build as normal. At the moment, the only packages you can add in this way are: ALUT, HGL, HUnit, OpenAL, OpenGL, QuickCheck, X11, cgi, fgl, haskell-src, html, mtl, network, parsec, time, xhtml because we're still using GHC's build system to build packages rather than Cabal. When we switch to using Cabal to build packages inside a GHC build, then it will be possible to populate the tree with any Cabal package. Possibly we could also distribution an "extra libs" source bundle containing these packages. - We will continue to build all these packages in the nightly builds. (but perhaps build failures in optional packages should not break the whole build). - darcs-all will have an option to grab the optional packages (all the packages in the list above). - Gentoo, Debian etc. will include only the core packages with GHC, the others will become separate entities. - We will continue to bundle the .tar.bz2 binary distributions, and the Windows installer, with all the optional packages. Perhaps there could also be minimal installers containing just the core packages. - We may have to change the set of core packages in the future, if GHC accumulates more dependencies for example. Any comments, or suggestions? Cheers, Simon
Simon Marlow wrote:
Here is what I propose to do regarding the libraries we ship with GHC 6.6. Comments are appreciated, since we need to finalise this story before the release candidate at the end of this week.
I have a small comment...
So here's what I propose:
- A "GHC source tree" will contain a core set of packages. This is what you will get if you download ghc-6.6-src.tar.bz2, or do "sh darcs-all get" in a darcs repo. The core packages are:
base, haskell98, template-haskell, readline, stm, Cabal, unix, Win32 plus hopefully regex-base and regex-posix if I integrate them this week. These are the packages required to bootstrap GHC, or those with deep GHC dependencies (template-haskell & stm).
You will also want regex-compat if you want to keep the old Text.Regex module API. (Renaming Text.Regex.New to Text.Regex) I have just gone over the stable repository [1] for regex-base,posix,compat and tweaked the cabal files to better match the packages they actually depend upon. The latest version of these is now 0.71 But I suspect you don't use cabal to build these for GHC. The posix c library is slow, but at least it is what people have been using. Note that Text.Regex.New from regex-compat is so close to Text.Regex that it shares the same "bugs", such as splitting or substituting with a pattern that matches an empty string and going into an infinite loop. Should I sacrifice a little backwards compatibility and catch these "bugs"? -- Chris [1] http://evenmere.org/~chrisk/trl/stable/ [2] http://evenmere.org/~chrisk/trl/head/
Chris Kuklewicz wrote:
Simon Marlow wrote:
Here is what I propose to do regarding the libraries we ship with GHC 6.6. Comments are appreciated, since we need to finalise this story before the release candidate at the end of this week.
I have a small comment...
So here's what I propose:
- A "GHC source tree" will contain a core set of packages. This is what you will get if you download ghc-6.6-src.tar.bz2, or do "sh darcs-all get" in a darcs repo. The core packages are:
base, haskell98, template-haskell, readline, stm, Cabal, unix, Win32 plus hopefully regex-base and regex-posix if I integrate them this week. These are the packages required to bootstrap GHC, or those with deep GHC dependencies (template-haskell & stm).
You will also want regex-compat if you want to keep the old Text.Regex module API. (Renaming Text.Regex.New to Text.Regex)
I have just gone over the stable repository [1] for regex-base,posix,compat and tweaked the cabal files to better match the packages they actually depend upon. The latest version of these is now 0.71
Is it ok if I take this code and turn it into separate repositories on darcs.haskell.org? Would you like to adopt the new repositories as the canonical location? I can set you up with a darcs.haskell.org account, just send me an SSH public key. How is the Haddock documentation? I noticed some of the modules didn't have Haddock headers on, this is standard practice in the libraries.
But I suspect you don't use cabal to build these for GHC.
The posix c library is slow, but at least it is what people have been using. Note that Text.Regex.New from regex-compat is so close to Text.Regex that it shares the same "bugs", such as splitting or substituting with a pattern that matches an empty string and going into an infinite loop.
Should I sacrifice a little backwards compatibility and catch these "bugs"?
I'd say that would be fine, replacing _|_ in a library API with something useful can't be a bad thing (unless you do it non-deterministically ;-). Cheers, Simon
Simon Marlow wrote:
Chris Kuklewicz wrote:
Simon Marlow wrote:
Here is what I propose to do regarding the libraries we ship with GHC 6.6. Comments are appreciated, since we need to finalise this story before the release candidate at the end of this week.
I have a small comment...
So here's what I propose:
- A "GHC source tree" will contain a core set of packages. This is what you will get if you download ghc-6.6-src.tar.bz2, or do "sh darcs-all get" in a darcs repo. The core packages are:
base, haskell98, template-haskell, readline, stm, Cabal, unix, Win32 plus hopefully regex-base and regex-posix if I integrate them this week. These are the packages required to bootstrap GHC, or those with deep GHC dependencies (template-haskell & stm).
You will also want regex-compat if you want to keep the old Text.Regex module API. (Renaming Text.Regex.New to Text.Regex)
I have just gone over the stable repository [1] for regex-base,posix,compat and tweaked the cabal files to better match the packages they actually depend upon. The latest version of these is now 0.71
Is it ok if I take this code and turn it into separate repositories on darcs.haskell.org? Would you like to adopt the new repositories as the canonical location? I can set you up with a darcs.haskell.org account, just send me an SSH public key.
Of course it is okay. All the regex-* packages are in a single darcs repository right now. Do you need a separate repository which only includes the parts that GHC will include? I am still gaining proficiency with darcs, and so I am not sure if the canonical repositories should be separate trees or not. The cabal packages all depend on (fps >= 0.7) to supply Data.ByteString. Currently there are: regex-base (everything below depends on this) regex-compat (depends on regex-posix and replaces Text.Regex) regex-posix (depends in regex in posix c library, must supply for Windows) regex-pcre (depends on libpcre, which is BSD) regex-tre (depends in libtre, which is LGPL) regex-parsec (no c dependency, but needs Parsec) regex-dfa (no c dependency, but needs Parsec) (Note: Engine is LGPL) regex-devel (depends on everything above) I will email you an SSH public key.
How is the Haddock documentation? I noticed some of the modules didn't have Haddock headers on, this is standard practice in the libraries.
I have not added the boilerplate haskell headers to everything. I will go through them and add it, probably tonight, to at least regex-base/posix/compat.
But I suspect you don't use cabal to build these for GHC.
The posix c library is slow, but at least it is what people have been using. Note that Text.Regex.New from regex-compat is so close to Text.Regex that it shares the same "bugs", such as splitting or substituting with a pattern that matches an empty string and going into an infinite loop.
Should I sacrifice a little backwards compatibility and catch these "bugs"?
I'd say that would be fine, replacing _|_ in a library API with something useful can't be a bad thing (unless you do it non-deterministically ;-).
Okay, I will feel free to remove a few _|_ conditions.
Chris Kuklewicz wrote:
Simon Marlow wrote:
Chris Kuklewicz wrote:
Simon Marlow wrote:
Here is what I propose to do regarding the libraries we ship with GHC 6.6. Comments are appreciated, since we need to finalise this story before the release candidate at the end of this week.
I have a small comment...
So here's what I propose:
- A "GHC source tree" will contain a core set of packages. This is what you will get if you download ghc-6.6-src.tar.bz2, or do "sh darcs-all get" in a darcs repo. The core packages are:
base, haskell98, template-haskell, readline, stm, Cabal, unix, Win32 plus hopefully regex-base and regex-posix if I integrate them this week. These are the packages required to bootstrap GHC, or those with deep GHC dependencies (template-haskell & stm).
You will also want regex-compat if you want to keep the old Text.Regex module API. (Renaming Text.Regex.New to Text.Regex)
I have just gone over the stable repository [1] for regex-base,posix,compat and tweaked the cabal files to better match the packages they actually depend upon. The latest version of these is now 0.71
Is it ok if I take this code and turn it into separate repositories on darcs.haskell.org? Would you like to adopt the new repositories as the canonical location? I can set you up with a darcs.haskell.org account, just send me an SSH public key.
Of course it is okay. All the regex-* packages are in a single darcs repository right now. Do you need a separate repository which only includes the parts that GHC will include? I am still gaining proficiency with darcs, and so I am not sure if the canonical repositories should be separate trees or not.
The cabal packages all depend on (fps >= 0.7) to supply Data.ByteString. Currently there are: regex-base (everything below depends on this) regex-compat (depends on regex-posix and replaces Text.Regex) regex-posix (depends in regex in posix c library, must supply for Windows) regex-pcre (depends on libpcre, which is BSD) regex-tre (depends in libtre, which is LGPL) regex-parsec (no c dependency, but needs Parsec) regex-dfa (no c dependency, but needs Parsec) (Note: Engine is LGPL) regex-devel (depends on everything above)
I will email you an SSH public key.
I need to add some build system stuff for GHC: Makefile, package.conf.in, and possibly configure.ac. Instead of the fps dependency, there will be a base>=2.0 dependency (ideally we would have conditional dependencies, but Cabal support for that isn't ready yet). Since time is short, I plan to integrate regex-base, regex-compat and regex-posix today, then I'll pull over any changes you make. I'll do this by replicating your trl-stable darcs repo and morphing it into each of the smaller repos, that way we don't lose history, and the repo isn't large to begin with.
How is the Haddock documentation? I noticed some of the modules didn't have Haddock headers on, this is standard practice in the libraries.
I have not added the boilerplate haskell headers to everything. I will go through them and add it, probably tonight, to at least regex-base/posix/compat.
A lot of the documentation seems to refer to JRegex - while this is useful for someone moving from one to the other, it won't mean much to someone trying to grok the new regex stuff in GHC 6.6. Any chance of having some more basic docs? Cheers, Simon
I need to add some build system stuff for GHC: Makefile, package.conf.in, and possibly configure.ac. Instead of the fps dependency, there will be a base>=2.0 dependency (ideally we would have conditional dependencies, but Cabal support for that isn't ready yet).
I don't have GHC past 6.4.2, so I'll have to let you do all that.
Since time is short, I plan to integrate regex-base, regex-compat and regex-posix today, then I'll pull over any changes you make. I'll do this by replicating your trl-stable darcs repo and morphing it into each of the smaller repos, that way we don't lose history, and the repo isn't large to begin with.
"morph"? The oldest version of the repository was "Text.Regex.Lazy" and was an unorganized mix of compat/parsec/dfa/devel without any base/posix/pcre/tre. The loss of history a this stage would not be too terrible.
How is the Haddock documentation? I noticed some of the modules didn't have Haddock headers on, this is standard practice in the libraries.
I have not added the boilerplate haskell headers to everything. I will go through them and add it, probably tonight, to at least regex-base/posix/compat.
A lot of the documentation seems to refer to JRegex - while this is useful for someone moving from one to the other, it won't mean much to someone trying to grok the new regex stuff in GHC 6.6. Any chance of having some more basic docs?
I should absorb the example files into a large piece of haddock documentation. I'll start on that and then merge that into your new repository. Cheers, Chris
On 23 August 2006 14:27, Chris Kuklewicz wrote:
I need to add some build system stuff for GHC: Makefile, package.conf.in, and possibly configure.ac. Instead of the fps dependency, there will be a base>=2.0 dependency (ideally we would have conditional dependencies, but Cabal support for that isn't ready yet).
I don't have GHC past 6.4.2, so I'll have to let you do all that.
No problem, I've already started but didn't have time to finish so far. Hopefully tomorrow.
Since time is short, I plan to integrate regex-base, regex-compat and regex-posix today, then I'll pull over any changes you make. I'll do this by replicating your trl-stable darcs repo and morphing it into each of the smaller repos, that way we don't lose history, and the repo isn't large to begin with.
"morph"?
As in, just move the files around so that the repo turns into just regex-base, and similarly for regex-posix etc.
The oldest version of the repository was "Text.Regex.Lazy" and was an unorganized mix of compat/parsec/dfa/devel without any base/posix/pcre/tre.
The loss of history a this stage would not be too terrible.
Should I just create fresh repos then? I don't mind either way. Cheers, Simon
Simon Marlow wrote:
On 23 August 2006 14:27, Chris Kuklewicz wrote:
I need to add some build system stuff for GHC: Makefile, package.conf.in, and possibly configure.ac. Instead of the fps dependency, there will be a base>=2.0 dependency (ideally we would have conditional dependencies, but Cabal support for that isn't ready yet).
I don't have GHC past 6.4.2, so I'll have to let you do all that.
No problem, I've already started but didn't have time to finish so far. Hopefully tomorrow.
I just pushed a (head and stable) version of regex-base,posix,compat with more boilerplate haddock and more comments. The Examples have not been added, but it is getting better.
Since time is short, I plan to integrate regex-base, regex-compat and regex-posix today, then I'll pull over any changes you make. I'll do this by replicating your trl-stable darcs repo and morphing it into each of the smaller repos, that way we don't lose history, and the repo isn't large to begin with. "morph"?
As in, just move the files around so that the repo turns into just regex-base, and similarly for regex-posix etc.
The oldest version of the repository was "Text.Regex.Lazy" and was an unorganized mix of compat/parsec/dfa/devel without any base/posix/pcre/tre.
The loss of history a this stage would not be too terrible.
Should I just create fresh repos then? I don't mind either way.
"Since time is short", just do whichever is fastest. If you convert and separate all the packages, then I suggest the top level Example*.hs and Makefile be kept as the top level of the separated regex-devel package. Otherwise I expect that I can convert it like that after I get an account. -- Chris
On Tue, 22 Aug 2006, Simon Marlow wrote:
- A source tree can optionally be populated with more packages, which will be included in the build as normal. At the moment, the only packages you can add in this way are:
ALUT, HGL, HUnit, OpenAL, OpenGL, QuickCheck, X11, cgi, fgl, haskell-src, html, mtl, network, parsec, time, xhtml
because we're still using GHC's build system to build packages rather than Cabal. When we switch to using Cabal to build packages inside a GHC build, then it will be possible to populate the tree with any Cabal package.
Possibly we could also distribution an "extra libs" source bundle containing these packages.
Just a question: Will it still be possible to have an Haddock index of _all_ installed libraries without hassle? I know haddock supports this, but how easy will it be to achieve that?
On 24 August 2006 11:20, Henning Thielemann wrote:
On Tue, 22 Aug 2006, Simon Marlow wrote:
- A source tree can optionally be populated with more packages, which will be included in the build as normal. At the moment, the only packages you can add in this way are:
ALUT, HGL, HUnit, OpenAL, OpenGL, QuickCheck, X11, cgi, fgl, haskell-src, html, mtl, network, parsec, time, xhtml
because we're still using GHC's build system to build packages rather than Cabal. When we switch to using Cabal to build packages inside a GHC build, then it will be possible to populate the tree with any Cabal package.
Possibly we could also distribution an "extra libs" source bundle containing these packages.
Just a question: Will it still be possible to have an Haddock index of _all_ installed libraries without hassle? I know haddock supports this, but how easy will it be to achieve that?
Since the distributions we build with the nightly builds will have all the libraries, the Haddock docs in the distributions we ship will have complete integrated indices. This means that making a minimal distribution in a build tree that has everything is not just a matter of removing stuff, also the Haddock index must be rebuilt, and the package database needs to have the extra packages removed too. Thanks for reminding me about this! I'm not sure we'll ship minimal binary installers for 6.6 yet, it depends whether we get to it before the release date. Cheers, Simon
participants (4)
-
Chris Kuklewicz -
Henning Thielemann -
Simon Marlow -
Simon Marlow