Hi everyone, As part of the recent GHC 8.2.1 release and the associated boot-library publishing to Hackage, I picked up again experimenting with a reinstallable lib:ghc package primarily for the benefit of uploading haddocks for lib:ghc (w/ `--hyperlinked-source`) to Hackage: http://hackage.haskell.org/package/ghc However, this also provides somewhat of a tech-preview of a reinstallable lib:ghc package, which would help mitigate a big issue we had in the past with lib:ghc and its dependencies being frozen to specific versions, and which in combination with cabal's nix-style pkg store that finally provide us with the ability to have multiple instances of the same library versions co-existing in the same package db, is no longer a requirement. So we could have lib:ghc more easily pick dependencies up (but still within reason!) like e.g. lib:text without forcing everyone to the single lib:text version bundled with GHC. And we'd also gain the ability to publish patch-level releases to lib:ghc only inbetween proper minor GHC releases (e.g. to support new versions of library dependencies). Consider this artificial example demo package which requires a different version of binary than the one bundled with GHC 8.2.1: --8<---------------cut here---------------start------------->8--- -- ghc-demo.cabal name: ghc-demo version: 0 build-type: Simple cabal-version: >=2.0 executable ghc-demo main-is: Main.hs build-depends: base ^>= 4.10, ghc ^>= 8.2.1, binary == 0.8.5.0 default-language: Haskell2010 --8<---------------cut here---------------end--------------->8--- Trying to build this will have the cabal solver complain that the installed instance of ghc requires a different (& installed) version of binary, i.e. binary-0.8.5.1. Moreover, the source version of lib:ghc on Hackage has the manual `buildable` cabal flag which defaults to false, and currently prevents lib:ghc to be reinstalled automatically (by enabling an unsatisfiable constraint): --8<---------------cut here---------------start------------->8--- Resolving dependencies... cabal: Could not resolve dependencies: trying: base-4.10.0.0/installed-4.1... (dependency of ghc-demo-0) next goal: ghc (dependency of ghc-demo-0) rejecting: ghc-8.2.1/installed-8.2... (conflict: binary==0.8.5.0, ghc => binary==0.8.5.1/installed-0.8...) trying: ghc-8.2.1 rejecting: ghc-8.2.1:-buildable (conflict: base==4.10.0.0/installed-4.1..., ghc -buildable => base<0) rejecting: ghc-8.2.1:+buildable (manual flag can only be changed explicitly) After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: ghc-demo, binary, base, ghc, ghc-8.2.1:buildable --8<---------------cut here---------------end--------------->8--- However, if we unlock this by toggling the flag (NB: this only works on linux/x86_64; it's explained later why), e.g. --8<---------------cut here---------------start------------->8--- -- cabal.project file packages: . package ghc flags: +buildable --8<---------------cut here---------------end--------------->8--- The cabal solver would come up with an install-plan: --8<---------------cut here---------------start------------->8--- Resolving dependencies... Build profile: -w ghc-8.2.1 -O1 In order, the following will be built (use -v for more details): - binary-0.8.5.0 {binary-0.8.5.0-7f686cf5c40c3c685540306709c4a2a30d740679f5d31ad196283d242a4d70ac} (lib) (requires download & build) - ghc-boot-8.2.1 {ghc-boot-8.2.1-7930c3d78690c5fa476415d38abbb7717b05cf65e80cc79c12e782c9886507cb} (lib) (requires build) - ghci-8.2.1 {ghci-8.2.1-6292924619a045fd18e9fde0cfa350632672ad7439cb84a5c56ff0e4781346b1} (lib) (requires build) - ghc-8.2.1 {ghc-8.2.1-bc10931b0f169622deb74d84cf10e315730dfd8f2edcf4cad9073d142cf5da0c} (lib) +buildable (requires build) - ghc-demo-0 {ghc-demo-0-inplace-ghc-demo} (exe:ghc-demo) (first run) Configuring binary-0.8.5.0-7f686cf5c40c3c685540306709c4a2a30d740679f5d31ad196283d242a4d70ac (lib) Building binary-0.8.5.0-7f686cf5c40c3c685540306709c4a2a30d740679f5d31ad196283d242a4d70ac (lib) Installing binary-0.8.5.0-7f686cf5c40c3c685540306709c4a2a30d740679f5d31ad196283d242a4d70ac (lib) Finished binary-0.8.5.0-7f686cf5c40c3c685540306709c4a2a30d740679f5d31ad196283d242a4d70ac (lib) Configuring ghc-boot-8.2.1-7930c3d78690c5fa476415d38abbb7717b05cf65e80cc79c12e782c9886507cb (lib) Building ghc-boot-8.2.1-7930c3d78690c5fa476415d38abbb7717b05cf65e80cc79c12e782c9886507cb (lib) Installing ghc-boot-8.2.1-7930c3d78690c5fa476415d38abbb7717b05cf65e80cc79c12e782c9886507cb (lib) Finished ghc-boot-8.2.1-7930c3d78690c5fa476415d38abbb7717b05cf65e80cc79c12e782c9886507cb (lib) Configuring ghci-8.2.1-6292924619a045fd18e9fde0cfa350632672ad7439cb84a5c56ff0e4781346b1 (lib) Building ghci-8.2.1-6292924619a045fd18e9fde0cfa350632672ad7439cb84a5c56ff0e4781346b1 (lib) Installing ghci-8.2.1-6292924619a045fd18e9fde0cfa350632672ad7439cb84a5c56ff0e4781346b1 (lib) Finished ghci-8.2.1-6292924619a045fd18e9fde0cfa350632672ad7439cb84a5c56ff0e4781346b1 (lib) Configuring ghc-8.2.1-bc10931b0f169622deb74d84cf10e315730dfd8f2edcf4cad9073d142cf5da0c (lib) Building ghc-8.2.1-bc10931b0f169622deb74d84cf10e315730dfd8f2edcf4cad9073d142cf5da0c (lib) Installing ghc-8.2.1-bc10931b0f169622deb74d84cf10e315730dfd8f2edcf4cad9073d142cf5da0c (lib) Finished ghc-8.2.1-bc10931b0f169622deb74d84cf10e315730dfd8f2edcf4cad9073d142cf5da0c (lib) Configuring executable 'ghc-demo' for ghc-demo-0.. Preprocessing executable 'ghc-demo' for ghc-demo-0.. Building executable 'ghc-demo' for ghc-demo-0.. [1 of 1] Compiling Main ( Main.hs, /tmp/lll/dist-newstyle/build/x86_64-linux/ghc-8.2.1/ghc-demo-0/c/ghc-demo/build/ghc-demo/ghc-demo-tmp/Main.o ) Linking /tmp/lll/dist-newstyle/build/x86_64-linux/ghc-8.2.1/ghc-demo-0/c/ghc-demo/build/ghc-demo/ghc-demo ... --8<---------------cut here---------------end--------------->8--- So, cabal was able to succesfully produce an executable which uses a reinstalled lib:ghc w/ a different lib:binary version! But now for the limitations: this currently works at best with a GHC 8.2.1 installation whose configuration matches the generated files I manually included in the modified lib:ghc package in http://hackage.haskell.org/package/ghc-8.2.1/src/autogen/ which as you can see from the generated files (see e.g. Config.hs) was a Linux/x86_64/libgmp/internal-libffi/... configuration. ...and this finally brings me to the purpose of why I wrote this email: In order to make lib:ghc properly reinstallable we'd need to either a) have a way to regenerate the files autogen/* via a custom Setup.hs (by the likes of genprimcode or similiar) or the lazy option, b) simply have those files installed in a place we can easily locate (again, from a custom Setup.hs) Does this make any sense; which option do you prefer? Also, I'd like to know if you can think of reasons why or situations when the reinstalled lib:ghc wouldn't work; or other reasons why this is a bad idea. Cheers, hvr On 2016-06-05 at 19:02:29 +0200, Herbert Valerio Riedel wrote:

...

As an ultimate goal for a project like this, it would be nice if you could do something like upgrade your base package but continue to use the ghc package without incompatible types.

That's one way to approach this. However, I've been experimenting with a more flexible way: Thanks to cabal-1.24's new nix-based tech-preview which gets rid of that `--force-reinstall` abomination for good, the prospect of simply recompiling the `ghc` cabal package becomes more attractive.

This would get rid of the primary reason which currently constraints us to using the package versions bundled in GHC (like e.g. for `bytestring`) as soon as `ghc` enters an install-plan.

I'm still investigating this, but early experiments were promising, but I need to reorganise GHC's source-tree a little bit to know for sure if this is feasable. But if this as I hope is possible, this could open up a lot of new possibilities!