Hi, this might only be tangentially relevant. However, you might consider this a working example of GHC and Cabal bleeding edge symbiosis. Some might have seen that I built some *relocatable* GHC releases for cross compilation (however those include the full base compiler for macOS and linux (deb8) as well) at http://hackage.mobilehaskell.org/. To facility those builds, I did not only need to make changes to GHC to allow it to detect its library folder relative to `ghc`. I also did made quite a lot of changes to hadrian, and cabal(!) to actually support describing GHC packages (primarily rts and ghc-prim) as cabal packages without resorting to hacks in ghc-cabal. And getting rid of ghc-cabal altogether. Any overly restrictive policy on GHC and Cabal releases will result in changes like theses to become a lot harder to do. After all a lot of work on GHC is performed by volunteers in their spare time, and I honestly can not see myself forcing someone else to spend even more time on streamlining project integrations, if even higher barriers are put into place. If we start dropping submodule dependencies, we need to make it trivial to put them back into place when hacking on GHC. I would therefore rather suggest we keep submodules in the development tree of GHC, but put a policy into place that says that submodules have to be replaced with their proper hackage dependencies (and versions) prior to any release. This would allow to keep working across ghc and multiple dependencies as source, while giving some additional non-in-flight guarantees about GHC releases and their dependencies. Another example would be my llvm-ng (bitcode producing llvm backend), which is naturally also a set of submodules in my ghc tree, as working on the code gen and ghc at the same time would either mean I need to fully integrate those libraries into GHC, or start bumping them on hackage for every minimal change while working on GHC and the code gen. By the end of the day, I believe we want to make hacking on GHC as easy for everyone as possible while at the same time improving the release quality. Cheers, Moritz

On Dec 18, 2017, at 7:03 PM, Boespflug, Mathieu wrote:

I think Mikhail's point is that if a package says build-type: Simple, then we know exactly what its Setup.hs says, and therefore also which part of the Cabal API it's using. Easy enough to keep that part stable even if others change. Case in point: Cabal-2.0 brought a number of changes to the overall API, but nothing that broke calling defaultMain from Distribution.Simple (which is what a build-type: Simple script does). At the end of the day the consumers of the wider Cabal API are pretty small. A substantial number of misc packages on Hackage do it but rarely heavily. Other than that it essentially comes down to Stack, cabal-install, Haskell For Mac and... any others?

My takeaway from the discussion so far is that the number of heavy consumers looks small enough that a draconian BC policy for Cabal-the-library sounds overkill, provided, crucially, that everything is in place, by GHC feature freeze at the very latest, to allow a smooth migration. A "smooth transition" means having a migration guide available before start of feature freeze etc, but to Ben's concern stated earlier in this thread (about GHC/upstream coupling), ideally also a release.

I should note that to the extent that GHC tracks upstream releases only (not git commits in unknown state), GHC can be released on a timely schedule without needing any coordination from upstream maintainers to await new releases on their part. So quite apart from the Cabal thing specifically, it's worth thinking about asking that the versions of all upstream packages only make it into GHC, at the behest of their respective maintainers, after a new release of upstream is made. This was already proposed earlier in the thread:

...

* [Proposal:] GHC does not track git commits of upstream dependencies in an unknown state of quality, but tracks vetted and tested releases instead.

Potentially, this could even mean drastically cutting down on the number of git submodules carried in the GHC repo. Since these packages could as well be downloaded from Hackage.

On 18 December 2017 at 05:04, Manuel M T Chakravarty wrote:

...

Yes, you are right Haskell for Mac also links against Cabal-the-library and API changes have regularly required me to fix my code. I guess, I have never been particularly stressed about it, because I also link against GHC API and that doesn’t even know how to spell API stability — i.e., changes required by Cabal are usually drowned out by the chaos inflicted by GHC.

In any case, you are making a good point.

Mikhail, I don’t understand your response to Mathieu at all. What does the build-type have to do with this?

Cheers, Manuel

...

15.12.2017, 19:41 Boespflug, Mathieu :

Thanks for the feedback, Michael.

Manuel, I believe you are also a Cabal-the-library consumer in Haskell For Mac?

Michael, you brought up another problem tangentially related to the original integer-gmp issue but that was not in my original list earlier in this thread:

* Cabal-2.0.0 had breaking changes in the API.

This means that by association GHC itself broke BC, because it shipped with Cabal-2.0, without the usual grace period.

Now, there are far fewer users of Cabal than of base. All, Michael in his previous email seems to be okay with breaking changes in Cabal given the conditions he stated (2 months grace period, advance notice of when the 2 months start). And perhaps this points to the lack of a need for the regular grace period applying to Cabal. How many other users of Cabal-the-library are there? In principle, every single Hackage package out there, which all have a Setup.hs script. Most of them are trivial, but how many did break because of these API changes? I for one am pretty happy for Cabal to move fast, but I'm concerned that these breaking changes happened without any kind of advance notice. To Simon's original point - there does not to be a clear policy and a good process surrounding Cabal itself and other GHC dependencies. So far we discussed mostly metadata changes, not API changes.

And to be clear, folks did get some (post facto) notice in September: http://coldwa.st/e/blog/2017-09-09-Cabal-2-0.html. That's helpful, but I submit that in the future this really should be part of the GHC release announcement (which happened over a month before that), and in fact a migration guide circulated before the feature freeze, so downstream tooling authors can adapt. If this is not possible, then perhaps it's premature for GHC to include that given Cabal release. Again, GHC should always have the option to stick to the old Cabal version until things get ironed out.

On 15 December 2017 at 08:42, Michael Snoyman wrote:

...

On Thu, Dec 14, 2017 at 12:27 PM, Boespflug, Mathieu wrote:

[snip]

...

* Or a middle ground: make feature freeze a thing. Meaning that for a couple of months before a major GHC release, the major new Cabal isn't technically released yet, but like GHC itself within this period, it's pretty staid, so not so much a moving target, and something downstream tooling authors can possibly adapt to even without any grace period on new metadata features. This assumes that the 2 months of feature freeze are enough time for downstream tooling. Thoughts from any of those maintainers?

Short answer: if there's a clear idea in advance of when this feature freeze is going to happen, I think we can coordinate releases of downstream tooling (Stack being the most important, but stackage-curator playing in as well) so that 2 months is sufficient. I'll talk with the rest of the Stack team to see if there are any concerns.

Longer answer: Stack intentionally avoids depending on the internals of Cabal wherever possible. Instead of calling library functions directly from within Haskell code to perform builds, for example, it interacts with the Setup.hs files over their command line interface.[1] This has two results:

* Stack can usually start using new GHC/Cabal versions without a new Stack release, since it's just shelling out for the actual build * There's not usually very much code churn needed in Stack to upgrade to a newer Cabal release

This past release was an exception because of all of the changes that landed, both the new cabal grammar to support the ^>= operator (making the old parser incapable of lossily parsing new files) and API changes (I think mostly around Backpack, though there was some code cleanup as well). In particular, the main interface we need from Cabal—the package description data types and parser—changed significantly enough that it took significant effort to upgrade. There were also new features added (like sub libraries and foreign libraries) that weren't immediately supported by the old Stack version, and had to be manually added in.

Tying this up: generally upgrading to a new Cabal release should be fine, and the only concern I'd have is fitting it into a release schedule with Stack. The complications that could slow that down are:

* Changes to the command line interface that Stack uses (hopefully those are exceedingly rare) * Major overhauls to the Stack-facing API

Michael

[1] This allows for more reproducible builds of older snapshots, insuring that the exact same Cabal library is performing the builds

_______________________________________________ Ghc-devops-group mailing list Ghc-devops-group@haskell.org https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs