GHC version control

Friends A few weeks back, we had a lively debate about what version control system to use for GHC. In the light of that discussion we promised a revised proposal. This is it. (We also promised a proposal about the build system: the final version is here: http://hackage.haskell.org/trac/ghc/wiki/Design/BuildSystem) Constructive comments welcomed! Simon GHC Version Control proposal: Sept 2008 * We still want to move from Darcs to Git for GHC itself. The recent flurry of work on Darcs is very encouraging, but GHC is a particularly high-end client, and taking a direct dependency leads to stress at both ends. * We do not plan to change before the 6.10 release. Doing so would be destabilising at a time when we are trying to stabilise; and we need to devote all our active cycles to getting 6.10 out. Yes, that means it'll be more painful to move patches from the (Git) HEAD to the (Darcs) 6.10 branch, but we'll put up with that. We'll delay the change until the 6.10 branch seems to have settled down; i.e. roughly 6.10.2. * A very desirable goal is that it be possible to build GHC with only Git, rather than requiring every developer to actively use both Darcs and Git. But, many of the libraries are maintained by others (notably Cabal) who, for good reasons, do not want to move. In what follows the "Boot Libraries" are the ones required to build GHC. * So we propose the following: - The GHC repo will be in Git - Each Boot Library will (a) either be mastered in Git, with a read-only Darcs mirror (b) or be mastered in Darcs, with a read-only Git mirror (c) or be mastered in Darcs, with an occasional, manual process to copy a snapshot of the library from Darcs into GHC's Git repo. (Those Git files should be considered read-only.) - That means that if we want to modify a Darcs-mastered library we'll have to get the Darcs version, make the patch, test it, push it, and then the Git mirror will be right. Inconvenient, but we can live with that. We might even arrange it to be possible for super-developers to use the Darcs repo (rather than the mirror) direct from their tree. Ordinary developers can continue to be Git-only. - The same issue will arise for other people who want to modify a Git-mastered package. - Which is which will be decided on a case-by-case basis. The main criterion is: is GHC a passive client of the library, which is maintained by someone else (e.g. Cabal, containers), or is the library intimately coupled with GHC's inner workings (e.g. template-haskell)? If in doubt we should resolve ties in favour of darcs; we can always change our minds later, but changing and then changing back would be silly. * Our specific proposals for the master VCS for each boot library are: hsc2hs darcs haddock2 either: up to David Waern packages/array git packages/base git packages/base3-compat git packages/bytestring darcs packages/Cabal darcs packages/containers darcs packages/directory darcs packages/editline darcs packages/filepath darcs packages/ghc-prim git packages/haskell98 darcs packages/hpc either: up to Andy Gill packages/integer-gmp git packages/old-locale darcs packages/old-time darcs packages/packedstring darcs packages/pretty darcs packages/process git packages/random darcs packages/syb either: up to Utrecht packages/template-haskell git packages/unix git packages/Win32 git * Of these, probably the only contentious one is 'base', which is both very tightly coupled to GHC, but also used by nhc and Hugs. We humbly beg indulgence from Malcolm and Ross, and hope that the Darcs mirror be acceptable. They have kindly offered such indulgence! * array, process, unix, Win32 are also somewhat less obvious, but they are quite intimately connected to GHC

Simon Peyton-Jones wrote:
In what follows the "Boot Libraries" are the ones required to build GHC.
* So we propose the following: - The GHC repo will be in Git
- Each Boot Library will (a) either be mastered in Git, with a read-only Darcs mirror (b) or be mastered in Darcs, with a read-only Git mirror (c) or be mastered in Darcs, with an occasional, manual process to copy a snapshot of the library from Darcs into GHC's Git repo. (Those Git files should be considered read-only.)
For (b), will the Git-mirror be automatically updated or patches from darcs be pulled manually?
- That means that if we want to modify a Darcs-mastered library we'll have to get the Darcs version, make the patch, test it, push it, and then the Git mirror will be right. Inconvenient, but we can live with that. We might even arrange it to be possible for super-developers to use the Darcs repo (rather than the mirror) direct from their tree. Ordinary developers can continue to be Git-only. [..] * Our specific proposals for the master VCS for each boot library are: hsc2hs darcs haddock2 either: up to David Waern packages/array git packages/base git packages/base3-compat git packages/bytestring darcs packages/Cabal darcs packages/containers darcs packages/directory darcs packages/editline darcs packages/filepath darcs packages/ghc-prim git packages/haskell98 darcs packages/hpc either: up to Andy Gill packages/integer-gmp git packages/old-locale darcs packages/old-time darcs packages/packedstring darcs packages/pretty darcs packages/process git packages/random darcs packages/syb either: up to Utrecht packages/template-haskell git packages/unix git packages/Win32 git
You do not distinguish between (b) and (c) in that list. Personally, I think, every (b) library is a Bad Thing if the Git-mirror update is *automatic*. Why? The setup discourages the library developer from validating their library changes against the GHC build. In other words, every single library that is marked as "darcs" above should be used by ghc at arms length. That is, ghc should not use the bleeding edge development version, but the latest stable version (or something similar). When there is a new stable version, the person maintaining the GHC Git mirror of the library ought to validate GHC with the new library version and only then update the mirror (or the in-place copy in GHC's Git repo). If it is not possible to use a library's latest stable version, the library is too tightly coupled with GHC to live in a darcs repo. Manuel

Hello,
The mirroring idea seems complex and fragile. Is there a reason why
GHC has to depend on the repository versions of the "external" (i.e.,
the ones marked with 'darcs') packages? It seems to me that it should
be possible (and desirable!) to make GHC depend on particular released
versions of those packages, and so the source could be obtained in the
same way that it is for other Haskell programs (e.g., hackage). Then,
the revision control system used to maintain those packages would not
be important from the point of view of GHC.
-Iavor
On Mon, Sep 15, 2008 at 5:58 PM, Manuel M T Chakravarty
Simon Peyton-Jones wrote:
In what follows the "Boot Libraries" are the ones required to build GHC.
* So we propose the following: - The GHC repo will be in Git
- Each Boot Library will (a) either be mastered in Git, with a read-only Darcs mirror (b) or be mastered in Darcs, with a read-only Git mirror (c) or be mastered in Darcs, with an occasional, manual process to copy a snapshot of the library from Darcs into GHC's Git repo. (Those Git files should be considered read-only.)
For (b), will the Git-mirror be automatically updated or patches from darcs be pulled manually?
- That means that if we want to modify a Darcs-mastered library we'll have to get the Darcs version, make the patch, test it, push it, and then the Git mirror will be right. Inconvenient, but we can live with that. We might even arrange it to be possible for super-developers to use the Darcs repo (rather than the mirror) direct from their tree. Ordinary developers can continue to be Git-only.
[..]
* Our specific proposals for the master VCS for each boot library are: hsc2hs darcs haddock2 either: up to David Waern packages/array git packages/base git packages/base3-compat git packages/bytestring darcs packages/Cabal darcs packages/containers darcs packages/directory darcs packages/editline darcs packages/filepath darcs packages/ghc-prim git packages/haskell98 darcs packages/hpc either: up to Andy Gill packages/integer-gmp git packages/old-locale darcs packages/old-time darcs packages/packedstring darcs packages/pretty darcs packages/process git packages/random darcs packages/syb either: up to Utrecht packages/template-haskell git packages/unix git packages/Win32 git
You do not distinguish between (b) and (c) in that list. Personally, I think, every (b) library is a Bad Thing if the Git-mirror update is *automatic*. Why? The setup discourages the library developer from validating their library changes against the GHC build.
In other words, every single library that is marked as "darcs" above should be used by ghc at arms length. That is, ghc should not use the bleeding edge development version, but the latest stable version (or something similar). When there is a new stable version, the person maintaining the GHC Git mirror of the library ought to validate GHC with the new library version and only then update the mirror (or the in-place copy in GHC's Git repo). If it is not possible to use a library's latest stable version, the library is too tightly coupled with GHC to live in a darcs repo.
Manuel
_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

Iavor Diatchki wrote:
The mirroring idea seems complex and fragile. Is there a reason why GHC has to depend on the repository versions of the "external" (i.e., the ones marked with 'darcs') packages? It seems to me that it should be possible (and desirable!) to make GHC depend on particular released versions of those packages, and so the source could be obtained in the same way that it is for other Haskell programs (e.g., hackage). Then, the revision control system used to maintain those packages would not be important from the point of view of GHC.
That is basically what I meant to say, too. Only Iavor says it more clearly! Manuel
On Mon, Sep 15, 2008 at 5:58 PM, Manuel M T Chakravarty
wrote: Simon Peyton-Jones wrote:
In what follows the "Boot Libraries" are the ones required to build GHC.
* So we propose the following: - The GHC repo will be in Git
- Each Boot Library will (a) either be mastered in Git, with a read-only Darcs mirror (b) or be mastered in Darcs, with a read-only Git mirror (c) or be mastered in Darcs, with an occasional, manual process to copy a snapshot of the library from Darcs into GHC's Git repo. (Those Git files should be considered read-only.)
For (b), will the Git-mirror be automatically updated or patches from darcs be pulled manually?
- That means that if we want to modify a Darcs-mastered library we'll have to get the Darcs version, make the patch, test it, push it, and then the Git mirror will be right. Inconvenient, but we can live with that. We might even arrange it to be possible for super-developers to use the Darcs repo (rather than the mirror) direct from their tree. Ordinary developers can continue to be Git-only.
[..]
* Our specific proposals for the master VCS for each boot library are: hsc2hs darcs haddock2 either: up to David Waern packages/array git packages/base git packages/base3-compat git packages/bytestring darcs packages/Cabal darcs packages/containers darcs packages/directory darcs packages/editline darcs packages/filepath darcs packages/ghc-prim git packages/haskell98 darcs packages/hpc either: up to Andy Gill packages/integer-gmp git packages/old-locale darcs packages/old-time darcs packages/packedstring darcs packages/pretty darcs packages/process git packages/random darcs packages/syb either: up to Utrecht packages/template-haskell git packages/unix git packages/Win32 git
You do not distinguish between (b) and (c) in that list. Personally, I think, every (b) library is a Bad Thing if the Git-mirror update is *automatic*. Why? The setup discourages the library developer from validating their library changes against the GHC build.
In other words, every single library that is marked as "darcs" above should be used by ghc at arms length. That is, ghc should not use the bleeding edge development version, but the latest stable version (or something similar). When there is a new stable version, the person maintaining the GHC Git mirror of the library ought to validate GHC with the new library version and only then update the mirror (or the in-place copy in GHC's Git repo). If it is not possible to use a library's latest stable version, the library is too tightly coupled with GHC to live in a darcs repo.
Manuel
_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries
_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

Manuel M T Chakravarty wrote:
Iavor Diatchki wrote:
The mirroring idea seems complex and fragile. Is there a reason why GHC has to depend on the repository versions of the "external" (i.e., the ones marked with 'darcs') packages? It seems to me that it should be possible (and desirable!) to make GHC depend on particular released versions of those packages, and so the source could be obtained in the same way that it is for other Haskell programs (e.g., hackage). Then, the revision control system used to maintain those packages would not be important from the point of view of GHC.
That is basically what I meant to say, too. Only Iavor says it more clearly!
Which is basically suggesting that there should be no (b) packages, only (c) (using the classification from Simon's email below). I think that is probably ok, and desirable as it reduces the number of different scenarios that the GHC developer has to think about. For the (c) packages where the GHC git repo contains a snapshot of the package, I can imagine also checking out the darcs repo into the GHC tree in order to work on the library, and as long as git ignores the _darcs directory everything should work fine. We can push changes to darcs, and also commit the new snapshot with git in the same tree. Of course, we don't plan to do this often, because we'll be using released versions of these packages. But I can imagine it being useful sometimes: for example, we'll need to work on Cabal a bit for the new build system, but after that I hope we'll be more decoupled from it. Cheers, Simon
Manuel
On Mon, Sep 15, 2008 at 5:58 PM, Manuel M T Chakravarty
wrote: Simon Peyton-Jones wrote:
In what follows the "Boot Libraries" are the ones required to build GHC.
* So we propose the following: - The GHC repo will be in Git
- Each Boot Library will (a) either be mastered in Git, with a read-only Darcs mirror (b) or be mastered in Darcs, with a read-only Git mirror (c) or be mastered in Darcs, with an occasional, manual process to copy a snapshot of the library from Darcs into GHC's Git repo. (Those Git files should be considered read-only.)
For (b), will the Git-mirror be automatically updated or patches from darcs be pulled manually?
- That means that if we want to modify a Darcs-mastered library we'll have to get the Darcs version, make the patch, test it, push it, and then the Git mirror will be right. Inconvenient, but we can live with that. We might even arrange it to be possible for super-developers to use the Darcs repo (rather than the mirror) direct from their tree. Ordinary developers can continue to be Git-only.
[..]
* Our specific proposals for the master VCS for each boot library are: hsc2hs darcs haddock2 either: up to David Waern packages/array git packages/base git packages/base3-compat git packages/bytestring darcs packages/Cabal darcs packages/containers darcs packages/directory darcs packages/editline darcs packages/filepath darcs packages/ghc-prim git packages/haskell98 darcs packages/hpc either: up to Andy Gill packages/integer-gmp git packages/old-locale darcs packages/old-time darcs packages/packedstring darcs packages/pretty darcs packages/process git packages/random darcs packages/syb either: up to Utrecht packages/template-haskell git packages/unix git packages/Win32 git
You do not distinguish between (b) and (c) in that list. Personally, I think, every (b) library is a Bad Thing if the Git-mirror update is *automatic*. Why? The setup discourages the library developer from validating their library changes against the GHC build.
In other words, every single library that is marked as "darcs" above should be used by ghc at arms length. That is, ghc should not use the bleeding edge development version, but the latest stable version (or something similar). When there is a new stable version, the person maintaining the GHC Git mirror of the library ought to validate GHC with the new library version and only then update the mirror (or the in-place copy in GHC's Git repo). If it is not possible to use a library's latest stable version, the library is too tightly coupled with GHC to live in a darcs repo.
Manuel
_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries
_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries
_______________________________________________ Cvs-ghc mailing list Cvs-ghc@haskell.org http://www.haskell.org/mailman/listinfo/cvs-ghc
participants (4)
-
Iavor Diatchki
-
Manuel M T Chakravarty
-
Simon Marlow
-
Simon Peyton-Jones