Here is what I propose to do regarding the libraries we ship with GHC 6.6. Comments are appreciated, since we need to finalise this story before the release candidate at the end of this week. For a while we have had the goal of devolving as many of the libraries that ship with GHC as possible. With Cabal, this is now possible: you can download GHC and then install 3rd-party packages with little hassle, or upgrade libraries that came with your GHC installation. So the days of the monolithic GHC+libraries installations should perhaps come to an end. On the other hand, it's often quite useful to be able to install a single distribution and get a comprehensive set of libraries. The folks who package GHC for the deeply package-based OSs like Debian, Gentoo and *BSD would like GHC to come with a minimal set of libraries and install the others as separate packages. Another consideration is that it's useful for our nightly builds to build as many libraries as possible, so we can help catch bitrot. However, the default GHC build now takes a very long time mostly due to the huge set of libraries, and most people building GHC don't need to build eg. OpenGL by default. What's more, the maintainers of many of these libraries would like to be able to make releases independently of GHC. So here's what I propose: - A "GHC source tree" will contain a core set of packages. This is what you will get if you download ghc-6.6-src.tar.bz2, or do "sh darcs-all get" in a darcs repo. The core packages are: base, haskell98, template-haskell, readline, stm, Cabal, unix, Win32 plus hopefully regex-base and regex-posix if I integrate them this week. These are the packages required to bootstrap GHC, or those with deep GHC dependencies (template-haskell & stm). - The core packages will have a stable API across patchlevel releases of GHC (perhaps with the exception of Cabal). We will fork the darcs repos of the core packages where necessary. - A source tree can optionally be populated with more packages, which will be included in the build as normal. At the moment, the only packages you can add in this way are: ALUT, HGL, HUnit, OpenAL, OpenGL, QuickCheck, X11, cgi, fgl, haskell-src, html, mtl, network, parsec, time, xhtml because we're still using GHC's build system to build packages rather than Cabal. When we switch to using Cabal to build packages inside a GHC build, then it will be possible to populate the tree with any Cabal package. Possibly we could also distribution an "extra libs" source bundle containing these packages. - We will continue to build all these packages in the nightly builds. (but perhaps build failures in optional packages should not break the whole build). - darcs-all will have an option to grab the optional packages (all the packages in the list above). - Gentoo, Debian etc. will include only the core packages with GHC, the others will become separate entities. - We will continue to bundle the .tar.bz2 binary distributions, and the Windows installer, with all the optional packages. Perhaps there could also be minimal installers containing just the core packages. - We may have to change the set of core packages in the future, if GHC accumulates more dependencies for example. Any comments, or suggestions? Cheers, Simon
Hello Simon, Tuesday, August 22, 2006, 2:11:46 PM, you wrote:
- A source tree can optionally be populated with more packages, which will be included in the build as normal. At the moment, the only packages you can add in this way are:
ALUT, HGL, HUnit, OpenAL, OpenGL, QuickCheck, X11, cgi, fgl, haskell-src, html, mtl, network, parsec, time, xhtml
- We will continue to bundle the .tar.bz2 binary distributions, and the Windows installer, with all the optional packages.
i'm agree with all that you said except for this moment. although it's more to Sigbjorn as a builder of Windows installer, i think that monolithic GHC installations should exclude large and rarely used graphics/sound packages: ObjectIO, GLUT, HGL, OpenGL, OpenAL, X11, ALUT and possibly HaXml and instead include smaller and more "fundamental" packages: ByteString, regexps, Collections, Edisson, Filepath, Time, networking, web. This will allow to cut down size of installer at about 20% and at the same time improve support of "basic" clocks that's required in any program. i think that inclusion of large packages in installer - independent of how great they are - is a wrong decision. -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com
Hi
i think that inclusion of large packages in installer - independent of how great they are - is a wrong decision.
Just for reference, I did this with WinHugs, I created WinHugs and MinHugs - with the explicit goal that a student at a uni should be able to install MinHugs on their disk space without blowing their quota. The difference was about 2Mb vs 40Mb installed. Thanks Neil
Hello Neil, Tuesday, August 22, 2006, 5:04:58 PM, you wrote:
Just for reference, I did this with WinHugs, I created WinHugs and MinHugs - with the explicit goal that a student at a uni should be able to install MinHugs on their disk space without blowing their quota. The difference was about 2Mb vs 40Mb installed.
btw, now size of large installer (using zip compression) is 12 mb. using cabarc compression, it is reduced to 2.8 mb, and using lzma - to 2.1 mb. so even switching to MSI installer alone will allow to cut off several times size of WinHugs installer -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com
Simon Marlow wrote:
So here's what I propose:
- A "GHC source tree" will contain a core set of packages. This is what you will get if you download ghc-6.6-src.tar.bz2, or do "sh darcs-all get" in a darcs repo. The core packages are:
base, haskell98, template-haskell, readline, stm, Cabal, unix, Win32
Should mtl not also be included here too as it is a relatively small package but seems fundamental to monadic programming which is one of the major highlights of Haskell? Ie doesn't almost everyone use at least one of the mtl transformers in their code? Regards, Brian. -- Logic empowers us and Love gives us purpose. Yet still phantoms restless for eras long past, congealed in the present in unthought forms, strive mightily unseen to destroy us. http://www.metamilk.com
Hello Brian, Tuesday, August 22, 2006, 5:52:29 PM, you wrote:
"sh darcs-all get" in a darcs repo. The core packages are:
base, haskell98, template-haskell, readline, stm, Cabal, unix, Win32
Should mtl not also be included here too as it is a relatively small package but seems fundamental to monadic programming which is one of the major highlights of Haskell?
Ie doesn't almost everyone use at least one of the mtl transformers in their code?
the idea is not "what we all use", but "what a required to build ghc itself, or closely depend on ghc version". i instead may propose to remove unix/win32 here - i guess that these packages are rather independent on GHC version -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com
Bulat Ziganshin wrote:
Hello Brian,
Tuesday, August 22, 2006, 5:52:29 PM, you wrote:
"sh darcs-all get" in a darcs repo. The core packages are:
base, haskell98, template-haskell, readline, stm, Cabal, unix, Win32
Should mtl not also be included here too as it is a relatively small package but seems fundamental to monadic programming which is one of the major highlights of Haskell?
Ie doesn't almost everyone use at least one of the mtl transformers in their code?
the idea is not "what we all use", but "what a required to build ghc itself, or closely depend on ghc version". i instead may propose to remove unix/win32 here - i guess that these packages are rather independent on GHC version
Ok I see now, that the whole point is to avoid having to download everything every time the ghc version changes, so I withdraw my suggestion about including mtl in the core :-) Brian. -- Logic empowers us and Love gives us purpose. Yet still phantoms restless for eras long past, congealed in the present in unthought forms, strive mightily unseen to destroy us. http://www.metamilk.com
Hello Brian, Tuesday, August 22, 2006, 6:26:02 PM, you wrote:
the idea is not "what we all use", but "what a required to build ghc itself, or closely depend on ghc version". i instead may propose to remove unix/win32 here - i guess that these packages are rather independent on GHC version
Ok I see now, that the whole point is to avoid having to download everything every time the ghc version changes, so I withdraw my suggestion about including mtl in the core :-)
to be exact, problem is not downloading, but separating libraries from the core package so they can be developed and updated independently. fo example, you can update mtl from 2.0 to 3.0 version while using still ghc 6.6 you installed four years ago. now library versions are fixed at moment of major GHC release (6.4/6.6/..) and don't changed during year or two -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com
Hello Simon, one interesting question you not mentioned - whether it will be possible to include in GHC bundles non-BSD licensed libs and tools. although it seems more on behalf of packagers, it's still interesting to discuss here. one thing that i definitely will be glad to see is MissingH library -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com
Bulat Ziganshin wrote:
one interesting question you not mentioned - whether it will be possible to include in GHC bundles non-BSD licensed libs and tools. although it seems more on behalf of packagers, it's still interesting to discuss here. one thing that i definitely will be glad to see is MissingH library
Certainly our bundles do include non-BSD components, for example GMP and the readline package. There's no problem with this, the only requirement is that the unit for licensing is a package, so that users stand a chance of being able to properly comply with licenses by just knowing which packages they use. As for MissingH, I'd rather see those libraries be incorporated into more canonical packages. John has done a lot of great work there, but I think we shouldn't be pushing libraries named 'MissingH.*', if these are libraries with important functionality that we want to adopt then they should move into the hierarchy proper. Cheers, Simon
Simon Marlow wrote:
Here is what I propose to do regarding the libraries we ship with GHC 6.6. Comments are appreciated, since we need to finalise this story before the release candidate at the end of this week.
I have a small comment...
So here's what I propose:
- A "GHC source tree" will contain a core set of packages. This is what you will get if you download ghc-6.6-src.tar.bz2, or do "sh darcs-all get" in a darcs repo. The core packages are:
base, haskell98, template-haskell, readline, stm, Cabal, unix, Win32 plus hopefully regex-base and regex-posix if I integrate them this week. These are the packages required to bootstrap GHC, or those with deep GHC dependencies (template-haskell & stm).
You will also want regex-compat if you want to keep the old Text.Regex module API. (Renaming Text.Regex.New to Text.Regex) I have just gone over the stable repository [1] for regex-base,posix,compat and tweaked the cabal files to better match the packages they actually depend upon. The latest version of these is now 0.71 But I suspect you don't use cabal to build these for GHC. The posix c library is slow, but at least it is what people have been using. Note that Text.Regex.New from regex-compat is so close to Text.Regex that it shares the same "bugs", such as splitting or substituting with a pattern that matches an empty string and going into an infinite loop. Should I sacrifice a little backwards compatibility and catch these "bugs"? -- Chris [1] http://evenmere.org/~chrisk/trl/stable/ [2] http://evenmere.org/~chrisk/trl/head/
Chris Kuklewicz wrote:
Simon Marlow wrote:
Here is what I propose to do regarding the libraries we ship with GHC 6.6. Comments are appreciated, since we need to finalise this story before the release candidate at the end of this week.
I have a small comment...
So here's what I propose:
- A "GHC source tree" will contain a core set of packages. This is what you will get if you download ghc-6.6-src.tar.bz2, or do "sh darcs-all get" in a darcs repo. The core packages are:
base, haskell98, template-haskell, readline, stm, Cabal, unix, Win32 plus hopefully regex-base and regex-posix if I integrate them this week. These are the packages required to bootstrap GHC, or those with deep GHC dependencies (template-haskell & stm).
You will also want regex-compat if you want to keep the old Text.Regex module API. (Renaming Text.Regex.New to Text.Regex)
I have just gone over the stable repository [1] for regex-base,posix,compat and tweaked the cabal files to better match the packages they actually depend upon. The latest version of these is now 0.71
Is it ok if I take this code and turn it into separate repositories on darcs.haskell.org? Would you like to adopt the new repositories as the canonical location? I can set you up with a darcs.haskell.org account, just send me an SSH public key. How is the Haddock documentation? I noticed some of the modules didn't have Haddock headers on, this is standard practice in the libraries.
But I suspect you don't use cabal to build these for GHC.
The posix c library is slow, but at least it is what people have been using. Note that Text.Regex.New from regex-compat is so close to Text.Regex that it shares the same "bugs", such as splitting or substituting with a pattern that matches an empty string and going into an infinite loop.
Should I sacrifice a little backwards compatibility and catch these "bugs"?
I'd say that would be fine, replacing _|_ in a library API with something useful can't be a bad thing (unless you do it non-deterministically ;-). Cheers, Simon
Simon Marlow wrote:
Chris Kuklewicz wrote:
Simon Marlow wrote:
Here is what I propose to do regarding the libraries we ship with GHC 6.6. Comments are appreciated, since we need to finalise this story before the release candidate at the end of this week.
I have a small comment...
So here's what I propose:
- A "GHC source tree" will contain a core set of packages. This is what you will get if you download ghc-6.6-src.tar.bz2, or do "sh darcs-all get" in a darcs repo. The core packages are:
base, haskell98, template-haskell, readline, stm, Cabal, unix, Win32 plus hopefully regex-base and regex-posix if I integrate them this week. These are the packages required to bootstrap GHC, or those with deep GHC dependencies (template-haskell & stm).
You will also want regex-compat if you want to keep the old Text.Regex module API. (Renaming Text.Regex.New to Text.Regex)
I have just gone over the stable repository [1] for regex-base,posix,compat and tweaked the cabal files to better match the packages they actually depend upon. The latest version of these is now 0.71
Is it ok if I take this code and turn it into separate repositories on darcs.haskell.org? Would you like to adopt the new repositories as the canonical location? I can set you up with a darcs.haskell.org account, just send me an SSH public key.
Of course it is okay. All the regex-* packages are in a single darcs repository right now. Do you need a separate repository which only includes the parts that GHC will include? I am still gaining proficiency with darcs, and so I am not sure if the canonical repositories should be separate trees or not. The cabal packages all depend on (fps >= 0.7) to supply Data.ByteString. Currently there are: regex-base (everything below depends on this) regex-compat (depends on regex-posix and replaces Text.Regex) regex-posix (depends in regex in posix c library, must supply for Windows) regex-pcre (depends on libpcre, which is BSD) regex-tre (depends in libtre, which is LGPL) regex-parsec (no c dependency, but needs Parsec) regex-dfa (no c dependency, but needs Parsec) (Note: Engine is LGPL) regex-devel (depends on everything above) I will email you an SSH public key.
How is the Haddock documentation? I noticed some of the modules didn't have Haddock headers on, this is standard practice in the libraries.
I have not added the boilerplate haskell headers to everything. I will go through them and add it, probably tonight, to at least regex-base/posix/compat.
But I suspect you don't use cabal to build these for GHC.
The posix c library is slow, but at least it is what people have been using. Note that Text.Regex.New from regex-compat is so close to Text.Regex that it shares the same "bugs", such as splitting or substituting with a pattern that matches an empty string and going into an infinite loop.
Should I sacrifice a little backwards compatibility and catch these "bugs"?
I'd say that would be fine, replacing _|_ in a library API with something useful can't be a bad thing (unless you do it non-deterministically ;-).
Okay, I will feel free to remove a few _|_ conditions.
Chris Kuklewicz wrote:
Simon Marlow wrote:
Chris Kuklewicz wrote:
Simon Marlow wrote:
Here is what I propose to do regarding the libraries we ship with GHC 6.6. Comments are appreciated, since we need to finalise this story before the release candidate at the end of this week.
I have a small comment...
So here's what I propose:
- A "GHC source tree" will contain a core set of packages. This is what you will get if you download ghc-6.6-src.tar.bz2, or do "sh darcs-all get" in a darcs repo. The core packages are:
base, haskell98, template-haskell, readline, stm, Cabal, unix, Win32 plus hopefully regex-base and regex-posix if I integrate them this week. These are the packages required to bootstrap GHC, or those with deep GHC dependencies (template-haskell & stm).
You will also want regex-compat if you want to keep the old Text.Regex module API. (Renaming Text.Regex.New to Text.Regex)
I have just gone over the stable repository [1] for regex-base,posix,compat and tweaked the cabal files to better match the packages they actually depend upon. The latest version of these is now 0.71
Is it ok if I take this code and turn it into separate repositories on darcs.haskell.org? Would you like to adopt the new repositories as the canonical location? I can set you up with a darcs.haskell.org account, just send me an SSH public key.
Of course it is okay. All the regex-* packages are in a single darcs repository right now. Do you need a separate repository which only includes the parts that GHC will include? I am still gaining proficiency with darcs, and so I am not sure if the canonical repositories should be separate trees or not.
The cabal packages all depend on (fps >= 0.7) to supply Data.ByteString. Currently there are: regex-base (everything below depends on this) regex-compat (depends on regex-posix and replaces Text.Regex) regex-posix (depends in regex in posix c library, must supply for Windows) regex-pcre (depends on libpcre, which is BSD) regex-tre (depends in libtre, which is LGPL) regex-parsec (no c dependency, but needs Parsec) regex-dfa (no c dependency, but needs Parsec) (Note: Engine is LGPL) regex-devel (depends on everything above)
I will email you an SSH public key.
I need to add some build system stuff for GHC: Makefile, package.conf.in, and possibly configure.ac. Instead of the fps dependency, there will be a base>=2.0 dependency (ideally we would have conditional dependencies, but Cabal support for that isn't ready yet). Since time is short, I plan to integrate regex-base, regex-compat and regex-posix today, then I'll pull over any changes you make. I'll do this by replicating your trl-stable darcs repo and morphing it into each of the smaller repos, that way we don't lose history, and the repo isn't large to begin with.
How is the Haddock documentation? I noticed some of the modules didn't have Haddock headers on, this is standard practice in the libraries.
I have not added the boilerplate haskell headers to everything. I will go through them and add it, probably tonight, to at least regex-base/posix/compat.
A lot of the documentation seems to refer to JRegex - while this is useful for someone moving from one to the other, it won't mean much to someone trying to grok the new regex stuff in GHC 6.6. Any chance of having some more basic docs? Cheers, Simon
I need to add some build system stuff for GHC: Makefile, package.conf.in, and possibly configure.ac. Instead of the fps dependency, there will be a base>=2.0 dependency (ideally we would have conditional dependencies, but Cabal support for that isn't ready yet).
I don't have GHC past 6.4.2, so I'll have to let you do all that.
Since time is short, I plan to integrate regex-base, regex-compat and regex-posix today, then I'll pull over any changes you make. I'll do this by replicating your trl-stable darcs repo and morphing it into each of the smaller repos, that way we don't lose history, and the repo isn't large to begin with.
"morph"? The oldest version of the repository was "Text.Regex.Lazy" and was an unorganized mix of compat/parsec/dfa/devel without any base/posix/pcre/tre. The loss of history a this stage would not be too terrible.
How is the Haddock documentation? I noticed some of the modules didn't have Haddock headers on, this is standard practice in the libraries.
I have not added the boilerplate haskell headers to everything. I will go through them and add it, probably tonight, to at least regex-base/posix/compat.
A lot of the documentation seems to refer to JRegex - while this is useful for someone moving from one to the other, it won't mean much to someone trying to grok the new regex stuff in GHC 6.6. Any chance of having some more basic docs?
I should absorb the example files into a large piece of haddock documentation. I'll start on that and then merge that into your new repository. Cheers, Chris
On 23 August 2006 14:27, Chris Kuklewicz wrote:
I need to add some build system stuff for GHC: Makefile, package.conf.in, and possibly configure.ac. Instead of the fps dependency, there will be a base>=2.0 dependency (ideally we would have conditional dependencies, but Cabal support for that isn't ready yet).
I don't have GHC past 6.4.2, so I'll have to let you do all that.
No problem, I've already started but didn't have time to finish so far. Hopefully tomorrow.
Since time is short, I plan to integrate regex-base, regex-compat and regex-posix today, then I'll pull over any changes you make. I'll do this by replicating your trl-stable darcs repo and morphing it into each of the smaller repos, that way we don't lose history, and the repo isn't large to begin with.
"morph"?
As in, just move the files around so that the repo turns into just regex-base, and similarly for regex-posix etc.
The oldest version of the repository was "Text.Regex.Lazy" and was an unorganized mix of compat/parsec/dfa/devel without any base/posix/pcre/tre.
The loss of history a this stage would not be too terrible.
Should I just create fresh repos then? I don't mind either way. Cheers, Simon
Simon Marlow wrote:
On 23 August 2006 14:27, Chris Kuklewicz wrote:
I need to add some build system stuff for GHC: Makefile, package.conf.in, and possibly configure.ac. Instead of the fps dependency, there will be a base>=2.0 dependency (ideally we would have conditional dependencies, but Cabal support for that isn't ready yet).
I don't have GHC past 6.4.2, so I'll have to let you do all that.
No problem, I've already started but didn't have time to finish so far. Hopefully tomorrow.
I just pushed a (head and stable) version of regex-base,posix,compat with more boilerplate haddock and more comments. The Examples have not been added, but it is getting better.
Since time is short, I plan to integrate regex-base, regex-compat and regex-posix today, then I'll pull over any changes you make. I'll do this by replicating your trl-stable darcs repo and morphing it into each of the smaller repos, that way we don't lose history, and the repo isn't large to begin with. "morph"?
As in, just move the files around so that the repo turns into just regex-base, and similarly for regex-posix etc.
The oldest version of the repository was "Text.Regex.Lazy" and was an unorganized mix of compat/parsec/dfa/devel without any base/posix/pcre/tre.
The loss of history a this stage would not be too terrible.
Should I just create fresh repos then? I don't mind either way.
"Since time is short", just do whichever is fastest. If you convert and separate all the packages, then I suggest the top level Example*.hs and Makefile be kept as the top level of the separated regex-devel package. Otherwise I expect that I can convert it like that after I get an account. -- Chris
participants (6)
-
Brian Hulley -
Bulat Ziganshin -
Chris Kuklewicz -
Neil Mitchell -
Simon Marlow -
Simon Marlow