[GHC] #16199: GHC 8.6's bundled version of transformers doesn't correspond to a Hackage release
#16199: GHC 8.6's bundled version of transformers doesn't correspond to a Hackage release -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: (none) Type: bug | Status: new Priority: highest | Milestone: Component: libraries | Version: 8.6.3 (other) | Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: #14678 Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- GHC 8.6 claims to be bundled with `transformers-0.5.5.0`, but this isn't true. That's because it's using transformers commit https://gitlab.haskell.org/ghc/packages/transformers/tree/80557845cdc0e72bc0..., which happened //after// the commit corresponding to the Hackage release of 0.5.5.0 (https://gitlab.haskell.org/ghc/packages/transformers/commit/33b3c8a71778ae37...). This is not the first time this has happened: see #14678. While we were able to catch #14678 before a major release was shipped, but unfortunately, that didn't happen this time. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16199 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#16199: GHC 8.6's bundled version of transformers doesn't correspond to a Hackage release -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: (none) Type: bug | Status: new Priority: highest | Milestone: Component: libraries | Version: 8.6.3 (other) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #14678 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by RyanGlScott): `process-1.6.3.0` is also affected: * The commit shipped with GHC 8.6 is https://gitlab.haskell.org/ghc/packages/process/commit/d14614572cc80bee734d4... * The Hackage commit for `1.6.3.0` is https://gitlab.haskell.org/ghc/packages/process/commit/7c0b58141290b50a338bf... -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16199#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#16199: GHC 8.6 bundles libraries that don't correspond to Hackage releases -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: (none) Type: bug | Status: new Priority: highest | Milestone: Component: libraries | Version: 8.6.3 (other) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #14678 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16199#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#16199: GHC 8.6 bundles libraries that don't correspond to Hackage releases -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: (none) Type: bug | Status: new Priority: highest | Milestone: Component: libraries | Version: 8.6.3 (other) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #14678 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Sigh, this is quite bad. We really need to rule this sort of thing out systematically from the release process. I'll work on this. In the meantime I have bumped `process` to 1.6.3.0 and `transformers` to 0.5.5.0 on `ghc-8.6`. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16199#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#16199: GHC 8.6 bundles libraries that don't correspond to Hackage releases -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: (none) Type: bug | Status: new Priority: highest | Milestone: Component: libraries | Version: 8.6.3 (other) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #14678 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by RyanGlScott): We should be careful about `transformers`, since there are actually API- relevant code changes that were introduced in commit https://gitlab.haskell.org/ghc/packages/transformers/commit/80557845cdc0e72b.... In particular, I'm pretty sure that `contravariant-1.5` requires these changes to compile (at least, angerman reported an error when he tried to build `contravariant-1.5` with the Hackage version of `transformers-0.5.5.0`). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16199#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#16199: GHC 8.6 bundles libraries that don't correspond to Hackage releases -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: (none) Type: bug | Status: new Priority: highest | Milestone: Component: libraries | Version: 8.6.3 (other) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #14678 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by angerman): So I've come up with a plan which I think should work. I'll try to whip it up today and see how it performs. I'll need the produces artifacts anyway to get this resolved. For each of the (external) libraries, we'll run `cabal new-sdist` (1) Then we'll pull the same one from hackage with `cabal get --pristine`. (2) And finally run a diff over the unpacked one from (1) and (2). This should yield an empty set. I'll try using this approach to generate me the sufficient patches for the packages that are in ghc and for which hackage has different source. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16199#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#16199: GHC 8.6 bundles libraries that don't correspond to Hackage releases -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: (none) Type: bug | Status: new Priority: highest | Milestone: Component: libraries | Version: 8.6.3 (other) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #14678 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by angerman): Alright, here we are (for ghc-8.6.3-release) {{{ -rw-r--r-- 1 angerman staff 0B Jan 18 10:00 Win32-2.6.1.0.patch -rw-r--r-- 1 angerman staff 0B Jan 18 10:00 array-0.5.3.0.patch -rw-r--r-- 1 angerman staff 642B Jan 18 10:00 binary-0.8.6.0.patch -rw-r--r-- 1 angerman staff 0B Jan 18 10:00 bytestring-0.10.8.2.patch -rw-r--r-- 1 angerman staff 70B Jan 18 10:00 containers-0.6.0.1.patch -rw-r--r-- 1 angerman staff 0B Jan 18 10:00 deepseq-1.4.4.0.patch -rw-r--r-- 1 angerman staff 0B Jan 18 10:00 directory-1.3.3.0.patch -rw-r--r-- 1 angerman staff 0B Jan 18 10:00 filepath-1.4.2.1.patch -rw-r--r-- 1 angerman staff 0B Jan 18 10:00 haskeline-0.7.4.3.patch -rw-r--r-- 1 angerman staff 675B Jan 18 10:00 hpc-0.6.0.3.patch -rw-r--r-- 1 angerman staff 0B Jan 18 10:00 mtl-2.2.2.patch -rw-r--r-- 1 angerman staff 0B Jan 18 10:00 parallel-3.2.2.0.patch -rw-r--r-- 1 angerman staff 1.4K Jan 18 10:00 parsec-3.1.13.0.patch -rw-r--r-- 1 angerman staff 0B Jan 18 10:00 pretty-1.1.3.6.patch -rw-r--r-- 1 angerman staff 6.0K Jan 18 10:00 process-1.6.3.0.patch -rw-r--r-- 1 angerman staff 0B Jan 18 10:00 stm-2.5.0.0.patch -rw-r--r-- 1 angerman staff 0B Jan 18 10:00 terminfo-0.4.1.2.patch -rw-r--r-- 1 angerman staff 0B Jan 18 10:00 text-1.2.3.1.patch -rw-r--r-- 1 angerman staff 5.4K Jan 18 10:00 time-1.8.0.2.patch -rw-r--r-- 1 angerman staff 20K Jan 18 10:00 transformers-0.5.5.0.patch -rw-r--r-- 1 angerman staff 4.8K Jan 18 10:00 unix-2.7.2.2.patch -rw-r--r-- 1 angerman staff 0B Jan 18 10:00 xhtml-3000.2.2.1.patch }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16199#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#16199: GHC 8.6 bundles libraries that don't correspond to Hackage releases -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: (none) Type: bug | Status: new Priority: highest | Milestone: Component: libraries | Version: 8.6.3 (other) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #14678 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by angerman): * Attachment "verify-packages.sh" added. verify-packages.sh -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16199 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#16199: GHC 8.6 bundles libraries that don't correspond to Hackage releases -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: (none) Type: bug | Status: new Priority: highest | Milestone: Component: libraries | Version: 8.6.3 (other) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #14678 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by angerman): The verify-packages.sh will generate the cabal sdists, pull the corresponding package from hackage, and diff them. Giving you the output above in the end. Ideally they should all be 0B in size. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16199#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#16199: GHC 8.6 bundles libraries that don't correspond to Hackage releases -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: (none) Type: bug | Status: patch Priority: highest | Milestone: Component: libraries | Version: 8.6.3 (other) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #14678 | Differential Rev(s): Wiki Page: | https://gitlab.haskell.org/ghc/ghc/merge_requests/139 -------------------------------------+------------------------------------- Changes (by angerman): * status: new => patch * differential: => https://gitlab.haskell.org/ghc/ghc/merge_requests/139 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16199#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#16199: GHC 8.6 bundles libraries that don't correspond to Hackage releases -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: (none) Type: bug | Status: patch Priority: highest | Milestone: Component: libraries | Version: 8.6.3 (other) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #14678 | Differential Rev(s): Wiki Page: | https://gitlab.haskell.org/ghc/ghc/merge_requests/139 -------------------------------------+------------------------------------- Comment (by hvr): Replying to [comment:5 angerman]:
For each of the (external) libraries, we'll run `cabal new-sdist` (1) Then we'll pull the same one from hackage with `cabal get --pristine`. (2) And finally run a diff over the unpacked one from (1) and (2).
This should yield an empty set.
Fwiw, we can't guarantee this in general, nor do we need to. There is no requirement that you can actually reinstall a boot package release bundled with GHC. There's no need for that, and there's packages that inevitably fall into this category. So tooling needs to be able to cope with that anyway; and in fact Cabal does. That being said, the only invariant that actually matters and the only one I care about very strongly is that of **consistent API versioning**: **If a package `foo-1.2.3` is bundled with GHC then if-and-only-if there is a valid build-plan (meeting all involved version constraints), then the `foo-1.2.3` from Hackage re-synthesized via Cabal must not to be distinguishable at the API level from the GHC pkg-db pre-installed one. In other words, `foo-1.2.3` shall uniquely determine the observable API exposed of that package.** A very prominent example was the `mtl` package that shipped with GHC 8.4.1 which broke the API versioning and caused real-world problems which (see https://github.com/haskell-hvr/HsYAML/issues/1 which required some very awkward CPP -- but in general this kind of problem is not always workaroundable by CPP! We ''must'' be able to trust the API versioning or we'd have to give up on version numbers altogether) Anything else, like diverging testsuites/benchmark components or other deltas which have no effect on the exposed API are of very little concern to me. Avoiding divergences would require enough time between the final release and the preceding release of a GHC last-beta or release-candidate after which boot package maintainers can be encouraged to publish their package release candidates on Hackage to the primary index (and I have to stress: "and no sooner!", as it's strongly discouraged to make releases on Hackage which target or advertise support for unreleased dependencies or tooling, or causing avoidable uploads to Hackage as is clearly specified in the Hackage upload guidelines), after we have ensured (in topological dep ordering) that the package candidates are sound and in sync with the artifacts included in the GHC release, and also fix any minor and not so minor issues detected in the process. or spelled out differently, the protocol for finalising boot libraries would be more or less like 1. release a last-beta/release-candidate 2. go over all boot libraries in topological sorted order and ensure their exposed API matched the Hackage release candidate; ensure that the compatibility advertised via version constraints is accurate; if needed, coordinate fixups with the maintainer 3. restart 2. in case there's not-so-minor changes which might cause an avalanche effect on their rev-deps 4. if there were fixups needed, make sure to `./validate` again; 5. when it's clear that the release-candidate has no more issues (otherwise go back to 1.) and we're going to cut the final release; encourage package maintainers to publish their latest Hackage package release candidate (which we verified to match the one contained in GHC's release) to the main index (i.e. press that "publish" button in the Hackage release candidate UI) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16199#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#16199: GHC 8.6 bundles libraries that don't correspond to Hackage releases -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: (none) Type: bug | Status: patch Priority: highest | Milestone: Component: libraries | Version: 8.6.3 (other) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #14678 | Differential Rev(s): Wiki Page: | https://gitlab.haskell.org/ghc/ghc/merge_requests/139 -------------------------------------+------------------------------------- Comment (by jrp): I don't know whether this is just regular `cabal` hell or an instance of this problem, but running: {{{ cabal new-install parsec Resolving dependencies... cabal: Could not resolve dependencies: [__0] trying: ghc-8.6.3/installed-8.6... (user goal) [__1] next goal: process (user goal) [__1] rejecting: process-1.6.5.0 (conflict: ghc => process==1.6.3.0/installed-1.6...) [__1] rejecting: process-1.6.4.0, process-1.6.3.0/installed-1.6..., process-1.6.3.0, process-1.6.2.0, process-1.6.1.0, process-1.6.0.0, process-1.5.0.0, process-1.4.3.0, process-1.4.2.0, process-1.4.1.0, process-1.4.0.0, process-1.3.0.0, process-1.2.3.0, process-1.2.2.0, process-1.2.1.0, process-1.2.0.0, process-1.1.0.2, process-1.1.0.1, process-1.1.0.0, process-1.0.1.5, process-1.0.1.4, process-1.0.1.3, process-1.0.1.2, process-1.0.1.1, process-1.0.0.0 (constraint from user target requires ==1.6.5.0) [__1] fail (backjumping, conflict set: ghc, process) After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: process, ghc }}} If an instance of this problem, then I'd recommend an 8.6.4... -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16199#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#16199: GHC 8.6 bundles libraries that don't correspond to Hackage releases -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: (none) Type: bug | Status: patch Priority: highest | Milestone: Component: libraries | Version: 8.6.3 (other) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #14678 | Differential Rev(s): Wiki Page: | https://gitlab.haskell.org/ghc/ghc/merge_requests/139 -------------------------------------+------------------------------------- Comment (by RyanGlScott): https://gitlab.haskell.org/ghc/ghc/merge_requests/139 was merged. Should we close this ticket, or would it behoove us to keep it open until we can verify that 8.6.4 fixes this issue? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16199#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#16199: GHC 8.6 bundles libraries that don't correspond to Hackage releases -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: (none) Type: bug | Status: closed Priority: highest | Milestone: 8.6.4 Component: libraries | Version: 8.6.3 (other) | Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #14678 | Differential Rev(s): Wiki Page: | https://gitlab.haskell.org/ghc/ghc/merge_requests/139 -------------------------------------+------------------------------------- Changes (by bgamari): * status: patch => closed * resolution: => fixed * milestone: => 8.6.4 Comment: Let's close it. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16199#comment:12 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#16199: GHC 8.6 bundles libraries that don't correspond to Hackage releases -------------------------------------+------------------------------------- Reporter: RyanGlScott | Owner: (none) Type: bug | Status: closed Priority: highest | Milestone: 8.6.4 Component: libraries | Version: 8.6.3 (other) | Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #14678 | Differential Rev(s): Wiki Page: | https://gitlab.haskell.org/ghc/ghc/merge_requests/139 -------------------------------------+------------------------------------- Comment (by jrp): It would be great to get 8.6.4 out, if only to fix this bug, which is a real productivity killer as I seem to need to empty my `.ghc` and `.cabal` directories and reinstall the libraries needed for whatever my project using to get going. Unless someone knows of a better workaround. {{{ Resolving dependencies... cabal: Could not resolve dependencies: [__0] trying: directory-1.3.3.2 (user goal) [__1] next goal: ghc (user goal) [__1] rejecting: ghc-8.6.3/installed-8.6... (conflict: directory==1.3.3.2, ghc => directory==1.3.3.0/installed-1.3...) [__1] rejecting: ghc-8.6.1, ghc-8.4.3, ghc-8.4.1, ghc-8.2.2, ghc-8.2.1 (constraint from user target requires ==8.6.3) [__1] fail (backjumping, conflict set: directory, ghc) After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: directory, ghc }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16199#comment:13 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC