December 2017 - ghc-devs

GHC-API unloading from external interpreter
by Németh Boldizsár 15 Dec '17

15 Dec '17

Dear GHC Developers, I'm using the GHC API to load Haskell modules and extract their syntax tree. I need to generate code where TH is used (by default I do in-memory linking). I've run into some trouble with external interpreter. When -fexternal-interpreter is NOT set, I use Linker.unload to remove bytecode from the global linker before reloading modules. However it is not applicable when external interpreter is used and I could not find corresponding functionality in the GHCi module (where communication with external interpreter is implemented). I've recently created 2 tickets related to this question: https://ghc.haskell.org/trac/ghc/ticket/14576, https://ghc.haskell.org/trac/ghc/ticket/14577 Can someone help me with this problem? Best Regards, Boldizsár Németh

2 2

Re: Fwd: Release policies
by Gershom B 15 Dec '17

15 Dec '17

Mathieu: I think the points about better tooling for documenting the correct claims in the release process are well taken. Updating the release notes manually leaves way too much room for error. However, I think you are incorrect that GHC 8.2.1 and 8.2.2 did not have cabal-install 1.24 support. They did. it works with them. I used it! The fact that the cabal files for builtin libraries use Cabal-2 syntax does not cause a problem. I think elsewhere Mikhail has corrected your timeline a bit. The general point is that Cabal-the-library and ghc releases do tend to be in tandem. But it is also the case that this happens because they are often developed in a coupled fashion. This in not really different than other builtin libs to GHC. They are necessarily coupled to changes in the compiler, and so end up being released together. Our process for making sure they are actually uploaded is absolutely error-prone as is the process for documenting what occurs in release notes. But I don't think we can simply by fiat decouple this stuff -- rather, this needs to occur on a technical level first, before other policies can really even be considered. Onto the concrete ideas: > * GHC should never under any circumstance ship with an unreleased version > of any independently maintained dependency. Cabal is one such dependency. > This should hold true for anything else. We could just add that policy to > the Release Policy. Yes, this is a good policy. Sometimes in the drive for a release, GHC will run ahead of a dependencies' upload by a bit. (Iirc this happened at one point in the past for bytestring, but this was some years ago). It is always better to hold things by a few days to get the dependency released first. > * Stronger still, GHC should not switch to a new major release of a > dependency at any time during feature freeze ahead of a release. E.g. if > Cabal-3.0.0 ships before feature freeze for GHC-9.6, then maybe it's fair > game to include in GHC. But not if Cabal-3.0.0 hasn't shipped yet. I don't think this works, in terms of coupled dependencies. If/when Cabal-3 is developed it will almost certainly be in tandem with GHC support. > * GHC does have control over reinstallable packages (like text and > bytestring): GHC need not ship with the latest versions of these, if indeed > they introduce breaking changes that would contravene the 3-release policy. Right, but this won't occur, because those libs are already under the control of the libraries committee and abide by the policy. > * Because there are far fewer consumers of metadata than consumers of say > base, I think shorter lead time is reasonable. At the other extreme, it > could even be just the few months during feature freeze. Right. There's not a sufficient quantity of metadata-consuming downstream tooling to motivate a long lead. And furthermore, the examples I can think of -- tracking dependencies for notifications or graphs -- have no need of pulling this data out of ghc-builtin libraries anyway, as they're for exploring the _userland_ portion of the package world regardless. (The other key example -- stack -- can also choose to ignore the metadata of builtin libs without harm). The general motivation of making a "feature freeze" more of a "freeze all the moving parts, really" I do agree with. Having a real freeze is part of a better release process, and it should allow all the downstream consumers of everything more time to really catch up. This is just one instance of that need. > Finally, a question for discussion: > > * Hackage allows revising the metadata of an uploaded package even without > changing the version number. This happens routinely on Hackage today by the > Hackage trustees. Should this be permitted for packages whose release is > completely tied to that of GHC itself (like integer-gmp)? It is rare that this is needed, but the ability just served us well -- editing integer-gmp to remove the new syntax was very useful, as it let us fix up old stack nightlies. Like all revisions, these should be made with some care and thought, but since we've just seen why it was helpful, I'd hate to now say we can't do it again if some other unforseen circumstance crops up. Regards, Gershom > Hi Simon, > > feedback from downstream consumers of Cabal metadata (e.g. build tool > authors) will be particularly useful for the discussion here. Here are my > thoughts as a bystander. > > It's worth trying to identify what problems came up during the integer-gmp > incident in Trac #14558: > > * GHC 8.2.1 shipped with integer-gmp-1.0.1.0 but the release notes said > otherwise. > * GHC 8.2.1 shipped with Cabal-2.0.0.2, but specifically claimed in the > release notes that cabal-install-1.24 (and by implication any other build > tool based on Cabal-the-library version 1.24) was supported: "GHC 8.2 only > works with cabal-install version 1.24 or later. Please upgrade if you have > an older version of cabal-install." > * GHC 8.2.2 also claimed Cabal-1.24 support. > * GHC 8.2.1 was released in July 2017 with Cabal-2.0.0.2, a brand new major > release with breaking changes to the metadata format, without much lead > time for downstream tooling authors (like Stack) to adapt. > * But actually if we look at their respective release notes, GHC 8.2.1 was > relased in July 2017, even though the Cabal website claims that > Cabal-2.0.0.2 was released in August 2017 (see > https://www.haskell.org/cabal/download.html) So it looks like GHC didn't > just not give enough lead time about an upstream dependency it shipped > with, it shipped with an unreleased version of Cabal! > * Libraries that ship with GHC are usually also uploaded to Hackage, to > make the documentation easily accessible, but integer-gmp-1.0.1.0 was not > uploaded to Hackage until 4 months after the release. > * The metadata for integer-gmp-1.0.1.0 as uploaded to Hackage differed from > the metadata that was actually in the source tarball of GHC-8.2.1 and > GHC-8.2.2. > * The metadata for integer-gmp-1.0.1.0 as uploaded to Hackage included > Cabal-2.0 specific syntactic sugar, making the metadata unreadable using > any tooling that did not link against the Cabal-2.0.0.2 library (or any > later version). > * It so happened that one particular version of one particular downstream > build tool, Stack, had a bug, compounding the bad effects of the previous > point. But a new release has now been made, and in any case that's not a > problem for GHC to solve. So let's keep that out of the discussion here. > > So I suggest we discuss ways to eliminate or reduce the likelihood of any > of the above problems from occurring again. Here are some ideas: > > * GHC should never under any circumstance ship with an unreleased version > of any independently maintained dependency. Cabal is one such dependency. > This should hold true for anything else. We could just add that policy to > the Release Policy. > * Stronger still, GHC should not switch to a new major release of a > dependency at any time during feature freeze ahead of a release. E.g. if > Cabal-3.0.0 ships before feature freeze for GHC-9.6, then maybe it's fair > game to include in GHC. But not if Cabal-3.0.0 hasn't shipped yet. > * The 3-release backwards compat rule should apply in all circumstances. > That means major version bumps of any library GHC ships with, including > base, should not imply any breaking change in the API's of any such library. > * GHC does have control over reinstallable packages (like text and > bytestring): GHC need not ship with the latest versions of these, if indeed > they introduce breaking changes that would contravene the 3-release policy. > * Note: today, users are effectively tied to whatever version of the > packages ships with GHC (i.e. the "reinstallable" bit is problematic today > for various technical reasons). That's why a breaking change in bytestring > is technically a breaking change in GHC. > * The current release policy covers API stability, but what about metadata? > In the extreme, we could say a 3-release policy applies to metadata too. > Meaning, all metadata shipping with GHC now and in the next 2 releases > should be parseable by today's version of Cabal and downstream tooling. Is > such a long lead time necessary? That's for build tool authors to say, and > a point to negotiate with GHC devs. > * Because there are far fewer consumers of metadata than consumers of say > base, I think shorter lead time is reasonable. At the other extreme, it > could even be just the few months during feature freeze. > * The release notes bugs mentioned above and the lack of consistent upload > to Hackage are a symptom of lack of release automation, I suspect. That's > how to fix it, but we could also spell out in the Release Policy that GHC > libraries should all be on Hackage from the day of release. > > Finally, a question for discussion: > > * Hackage allows revising the metadata of an uploaded package even without > changing the version number. This happens routinely on Hackage today by the > Hackage trustees. Should this be permitted for packages whose release is > completely tied to that of GHC itself (like integer-gmp)? > > Best, > > Mathieu > > > On 13 December 2017 at 17:43, Simon Peyton Jones via ghc-devs < > ghc-devs(a)haskell.org> wrote: > >> Dear GHC devops group >> >> The conversation on Trac #14558 >> <https://ghc.haskell.org/trac/ghc/ticket/14558> suggests that we might >> want to consider reviewing GHC’s release policies >> <https://ghc.haskell.org/trac/ghc/wiki/WorkingConventions/Releases>. >> This email is to invite your input. >> >> The broad questions is this. We want GHC to serve the needs of all its >> users, including downstream tooling that uses GHC. What release policies >> will best support that goal? For example, we already ensure that GHC 8.4 >> can be compiled with 8.2 and 8.0. This imposes a slight tax on GHC >> development, but it means that users don't need to upgrade quite as often. >> (If the tempo of releases increases, we might want to increase the window.) >> >> Trac #14558 suggests that we might want to ensure the metadata on GHC’s >> built-in libraries is parsable with older Cabals. One possibility would be >> this: >> >> - Ensure that the Cabal metadata of non-reinstallable packages (e.g. >> integer-gmp) shipped with GHC be parsable by the Cabal versions shipped >> with the last two major GHC releases [i.e. have a sufficiently old >> cabal-version field]. That is, in general a new Cabal specification will >> need to be shipped with two GHC releases before GHC will use start using >> its features in non-reinstallable packages. >> - Upholding this policy won't always be possible. There may be cases >> (as is the case Hadrian for GHC 8.4) where the benefit of quickly >> introducing incompatible syntax outweighs the need for compatibility. In >> this (hopefully rare) case we would explicitly advertise the >> incompatibility in the release documentation, and give as much notice as >> possible to users to allow downstream tools to adapt. >> - For reinstallable packages, of which GHC is simply a client (like >> text or bytestring), we can’t reasonably enforce such a policy, because GHC >> devs have no control over what the maintainers of external core libraries >> put in their Cabal files. >> >> This is just a proposal. The narrow questions are these: >> >> - Would this be sufficient to deal with the concerns raised in #14558? >> - Is it necessary, ow would anything simpler be sufficient? >> - What costs would the policy impose on GHC development? >> - There may be matters of detail: e.g. is two releases the right grace >> period. Would one do? >> >> Both the broad question and the narrow ones are appropriate for the Devops >> group. >> >> Thanks! >> >> Simon >> >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs(a)haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> >> >

5 14

Cannot unsubscribe
by cheater00 cheater00 13 Dec '17

13 Dec '17

I went to http://mail.haskell.org/cgi-bin/mailman/options/ghc-devs logged in and clicked on "unsubscribe" and no unsubscription email shows up. What do i do to unsubscribe? Thanks

1 1

Long standing annoying issue in ghci
by cheater00 cheater00 13 Dec '17

13 Dec '17

Hi guys, this one seems to have gotten buried under more important things, but maybe someone would like to take a look at it none the less? I've been encountering it literally every day for the last 4 years. I sometimes find it difficult to find read the past commands without color coding the prompt so I can't really turn it off. It seems like a simple arithmetic issue somewhere in the readline implementation. https://ghc.haskell.org/trac/ghc/ticket/9364 Thanks a lot.

4 9

Re: [GHC] #14537: Do not link to msvcrt.dll
by Phyx 13 Dec '17

13 Dec '17

Add a -v3 to ghc to see what's happening. On Wed, Dec 13, 2017, 07:23 GHC <ghc-devs(a)haskell.org> wrote: > #14537: Do not link to msvcrt.dll > -------------------------------------+------------------------------------- > Reporter: johndoe | Owner: (none) > Type: feature request | Status: closed > Priority: normal | Milestone: > Component: Compiler | Version: 8.2.1 > (Linking) | > Resolution: invalid | Keywords: > Operating System: Windows | Architecture: > | Unknown/Multiple > Type of failure: Other | Test Case: > Blocked By: | Blocking: > Related Tickets: | Differential Rev(s): > Wiki Page: | > -------------------------------------+------------------------------------- > > Comment (by johndoe): > > Well, I tried to modify `specs` directly, but project failed to build: > I've got messagebox that `setup.exe` crashed. > That was on my local machine, but the same you can see on appveyor: > https://ci.appveyor.com/project/johnd0e/pandoc/build/1.0.79 > > I am not sure, maybe my `specs` file is not good: > https://github.com/johnd0e/pandoc/blob/master/specs > But it worked well with other project (not GHC). > > -- > Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14537#comment:12> > GHC <http://www.haskell.org/ghc/> > The Glasgow Haskell Compiler >

1 0

GHCi recompilation avoidance UI
by David Feuer 11 Dec '17

11 Dec '17

Changes in GHC 8.2.1 lead to a lot of recompilation, because GHCi now refuses to load optimized code unless -fobject-code (and optimization flags) are enabled. I propose the following slight modification to https://ghc.haskell.org/trac/ghc/ticket/13604#comment:48 1. Optimization flags (except -O0) imply -fobject-code. This ensures that GHC respects optimization flags regardless of --interactive. 2. Even when -fobject-code is on, :load *M will load M as bytecode. This provides the "escape hatch" from -fobject-code that you need to use debugging features, etc. 3. New -fignore-optim-changes and -fignore-hpc-changes (Phab:D4123) flags should enable users to put together object code and bytecode with diverse optimization levels/options and HPC options while still updating automatically based on source changes and whether profiling is enabled. Does this seem like a reasonable way forward? David Feuer

5 8

cvs-ghc mailing list dead?
by Steven Shaw 10 Dec '17

10 Dec '17

This email list <https://mail.haskell.org/mailman/listinfo/cvs-ghc> appears to be dead but I figured it might forward to ghc-devs <https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs>. Just checking...

1 0

Moving hadrian to submodule
by Ben Gamari 08 Dec '17

08 Dec '17

Hello everyone, A bit over a month ago we merged hadrian into the ghc tree as a subtree. Unfortunately, those working on Hadrian have found the subtree mechanism to provide a rather poor developer experience. Consequently, today I will be ripping out the subtree and replacing it with a submodule. After pulling the commit performing this change you will likely need to do the following to emplace the new submodule, $ git submodule update --init $ git -C hadrian checkout . Cheers, - Ben

3 2

unix/tests/user001
by Simon Peyton Jones 08 Dec '17

08 Dec '17

On Linux I get this validate failure all the time. What should I do? I've trained myself to ignore it, but that seems wrong Simon Unexpected failures: ../libraries/unix/tests/user001.run user001 [bad stdout] (normal) =====> user001(normal) 1 of 1 [0, 0, 0] cd "./user001.run" && "/5playpen/simonpj/HEAD/inplace/test spaces/ghc-stage2" -o user001 user001.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -dno-debug-output -package unix cd "./user001.run" && ./user001 Actual stdout output differs from expected: diff -uw "./user001.run/user001.stdout.normalised" "./user001.run/user001.run.stdout.normalised" --- ./user001.run/user001.stdout.normalised 2017-12-08 10:22:59.835931843 +0000 +++ ./user001.run/user001.run.stdout.normalised 2017-12-08 10:22:59.835931843 +0000 @@ -4,8 +4,8 @@ getEffectiveGroupID: OK getGroups: OK getEffectiveUserName: OK -getGroupEntryForID: OK -getGroupEntryForName: OK +getGroupEntryForID: ERROR: getGroupEntryForID: does not exist (no such group) +getGroupEntryForName: ERROR: getGroupEntryForID: does not exist (no such group) getAllGroupEntries: OK getUserEntryForID: OK getAllUserEntries: OK *** unexpected failure for user001(normal)

2 3

Can't push to haddock
by Simon Peyton Jones 07 Dec '17

07 Dec '17

I'm trying to push a patch that needs a supporting change to haddock. I've pushed the haddock change to the ghc-head branch of ssh://git@github.com/haskell/haddock.git, which is (according to 'packages') the relevant haddock upstream repo. But when I try to push the GHC patch, I get this message bash$ git push Counting objects: 45, done. Delta compression using up to 32 threads. Compressing objects: 100% (45/45), done. Writing objects: 100% (45/45), 27.56 KiB | 0 bytes/s, done. Total 45 (delta 43), reused 0 (delta 0) remote: performing commit message validations... remote: Commit message validation passed! remote: performing submodule-ref update validations... remote: Submodule update(s) detected in fa29df02a1b0b926afb2525a258172dcbf0ea460: remote: utils/haddock => 24841386cff6fdccc11accf9daa815c2c7444d65 remote: *FAIL* commit not found in submodule repo ('../haddock.git') remote: or not reachable from persistent branches remote: hooklet hooks/update.secondary.d/check-submodule-refs failed remote: hooks/update.secondary died remote: error: hook declined to update refs/heads/master To ssh://git@git.haskell.org/ghc.git ! [remote rejected] HEAD -> master (hook declined) error: failed to push some refs to 'ssh://git@git.haskell.org/ghc.git' simonpj@cam-05-unx:~/code/HEAD$ What's up? I have pushed the haddock commit! THanks Simon

1 1