PSA for Cabal 2.2 new-* users regarding .ghc.environment files

There is an important change in the cabal new- commands for 2.2 that the release docs should have highlighted more significantly. Cabal new-* commands now produce a .ghc.environment file by default. These files [1] are picked up by ghc and ghci automatically (since 8.0.1), and allow them to operate directly with the same package environment used by the new-* commands. This lets you, for example, run `ghci` in a project where you are using `new-build` and get the proper dependencies in scope. Herbert has written an experimental tool to make it easier to create and manipulate these environments [2]. However: there is a drawback (on which some discussion at [3] and [4]. In particular, there is not good information provided by ghc about when these files are picked up and used. So if you are admixing new-* commands and other commands for the time being, jumping between the two, or admixing ghc and ghcjs, etc., then you may run into unexpected behavior! [5] The simplest solution for now is to delete the local .ghc.environment file in those cases (i.e. where you're mixing commands and get unexpected behavior). A particular gotcha is that these files are picked up not just in the current directory but also in any parent directory. Cheers, Gershom [1] documented at https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/packages.htm... [2] https://github.com/hvr/cabal-env [3] https://github.com/haskell/cabal/issues/4542 [4] https://ghc.haskell.org/trac/ghc/ticket/13753 [5] Error messages may be like ".cabal/store/ghc-8.0.2/package.db/package.cache: openBinaryFile: does not exist (No such file or directory)" or complaints about missing inplace dependencies.

So if you are admixing new-* commands and other commands for the time being, jumping between the two, or admixing ghc and ghcjs, etc., then you may run into unexpected behavior! The simplest solution for now is to delete the local .ghc.environment file in those cases
Is `cabal exec` suitable for working around this issue?
If not, is there something like `cabal new-exec` planned for the future?
-- Dan Burton
On Sat, May 12, 2018 at 1:41 PM, Gershom B
There is an important change in the cabal new- commands for 2.2 that the release docs should have highlighted more significantly.
Cabal new-* commands now produce a .ghc.environment file by default. These files [1] are picked up by ghc and ghci automatically (since 8.0.1), and allow them to operate directly with the same package environment used by the new-* commands. This lets you, for example, run `ghci` in a project where you are using `new-build` and get the proper dependencies in scope. Herbert has written an experimental tool to make it easier to create and manipulate these environments [2].
However: there is a drawback (on which some discussion at [3] and [4]. In particular, there is not good information provided by ghc about when these files are picked up and used. So if you are admixing new-* commands and other commands for the time being, jumping between the two, or admixing ghc and ghcjs, etc., then you may run into unexpected behavior! [5] The simplest solution for now is to delete the local .ghc.environment file in those cases (i.e. where you're mixing commands and get unexpected behavior). A particular gotcha is that these files are picked up not just in the current directory but also in any parent directory.
Cheers, Gershom
[1] documented at https://downloads.haskell.org/~ghc/latest/docs/html/users_ guide/packages.html#package-environments [2] https://github.com/hvr/cabal-env [3] https://github.com/haskell/cabal/issues/4542 [4] https://ghc.haskell.org/trac/ghc/ticket/13753 [5] Error messages may be like ".cabal/store/ghc-8.0.2/package.db/package.cache: openBinaryFile: does not exist (No such file or directory)" or complaints about missing inplace dependencies. _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

On Sat, May 12, 2018 at 10:41 PM, Gershom B
In particular, there is not good information provided by ghc about when these files are picked up and used.
Fwiw, there's already an overdue patch up for that at https://phabricator.haskell.org/D4689 If there's enough demand, this could in theory even make it into a future GHC 8.4.3

Seems like that’d be a worthwhile cleanup / fix up release so this can be
more visible / less silent. :)
On Sat, May 12, 2018 at 6:11 PM Herbert Valerio Riedel
On Sat, May 12, 2018 at 10:41 PM, Gershom B
wrote: In particular, there is not good information provided by ghc about when these files are picked up and used.
Fwiw, there's already an overdue patch up for that at https://phabricator.haskell.org/D4689
If there's enough demand, this could in theory even make it into a future GHC 8.4.3 _______________________________________________ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users

Am Samstag, den 12.05.2018, 16:41 -0400 schrieb Gershom B:
There is an important change in the cabal new- commands for 2.2 that the release docs should have highlighted more significantly.
Cabal new-* commands now produce a .ghc.environment file by default. These files [1] are picked up by ghc and ghci automatically (since 8.0.1), and allow them to operate directly with the same package environment used by the new-* commands. This lets you, for example, run `ghci` in a project where you are using `new-build` and get the proper dependencies in scope. Herbert has written an experimental tool to make it easier to create and manipulate these environments [2].
I already had issues with this. I ran some custom tool, which was a Haskell script, in a directory where I had run cabal new-build before. As a result, this tool didn’t work, because some module it needed couldn’t be found. It took me a while to find out what the problem was. In my opinion, it is questionable that these GHC environment files are generated by cabal new-build and then influence even Haskell scripts that have nothing to do with the project at hand. All the best, Wolfgang

The whole point of them is to enable doing things that no tooling can
sanely automate, enabling the environment from those builds to be available
to do those things. Which is admittedly trying to do two incompatible
things at once (propagate the build environment sometimes but not always,
with no way to know which is intended); if you have a good answer for that
one, I suspect many people would like to hear it.
Stack has this a bit easier in that it can just expose its own environment,
but it's also by design and intent less flexible than cabal new-build:
reproducible builds means a static, predefined environment instead of the
dynamic but constrained one presented by cabal new-build. Each tool has its
own purpose and use cases; but while cabal new-build can produce frozen
configurations emulating stack, there's no reasonable way to get a dynamic
configuration from stack. The flip side of which is you need those
environment files to get access to new-build's current/latest environment.
It's definitely "wrong solution", but nobody has been able to figure out
what the right solution is yet, or at least not in a way that works the way
ghc does.
(Which points to the further issue that how ghc works is itself rather
limiting, thus stack and cabal trying to impose alternative solutions on
it. But changing that is an even more tangled hairball, starting with "what
exactly do you change it to?". Which might not have the same answer for
stack vs. cabal, to say nothing of other build tools that might be in use
e.g. in corporate in-house programming and that nobody outside them knows
anything about.)
On Sun, May 13, 2018 at 4:53 AM, Wolfgang Jeltsch
Am Samstag, den 12.05.2018, 16:41 -0400 schrieb Gershom B:
There is an important change in the cabal new- commands for 2.2 that the release docs should have highlighted more significantly.
Cabal new-* commands now produce a .ghc.environment file by default. These files [1] are picked up by ghc and ghci automatically (since 8.0.1), and allow them to operate directly with the same package environment used by the new-* commands. This lets you, for example, run `ghci` in a project where you are using `new-build` and get the proper dependencies in scope. Herbert has written an experimental tool to make it easier to create and manipulate these environments [2].
I already had issues with this. I ran some custom tool, which was a Haskell script, in a directory where I had run cabal new-build before. As a result, this tool didn’t work, because some module it needed couldn’t be found. It took me a while to find out what the problem was.
In my opinion, it is questionable that these GHC environment files are generated by cabal new-build and then influence even Haskell scripts that have nothing to do with the project at hand.
All the best, Wolfgang _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.
-- brandon s allbery kf8nh sine nomine associates allbery.b@gmail.com ballbery@sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net

Brandon Allbery wrote:
The whole point of them is to enable doing things that no tooling can sanely automate, enabling the environment from those builds to be available to do those things.
From my own perspective, I expect that changing the working directory does not change the behavior of ghci except for where it looks for
I think the complaint is not about the functionality at all, but about the fact that this environment information is picked up implicitly. This is exacerbated by the fact that the resulting diagnostic messages are poor. source code. This is somewhat broken by .ghci files, but one can opt out of these; ideally I'd like local .ghci files to be opt in, and only $HOME/.ghci be parsed by default. So that's what I would advocate: ghc's functionality to use these local (CWD-based) environment files should be disabled by default and enabled by a command line switch. It's unfortunate that this functionality has been present since ghc 8.0 but has gone unnoticed until now that cabal-install actually uses it, so cabal-install gets most of the blame. But at the very least it should be easy to opt out of this functionality. So https://ghc.haskell.org/trac/ghc/ticket/13753 will help. Cheers, Bertram

On 05/13/2018 05:39 AM, Brandon Allbery wrote:
The whole point of them is to enable doing things that no tooling can sanely automate, enabling the environment from those builds to be available to do those things. Which is admittedly trying to do two incompatible things at once (propagate the build environment sometimes but not always, with no way to know which is intended); if you have a good answer for that one, I suspect many people would like to hear it.
I use nix-shell for choosing specific ghc environments, and undoubtedly a theoretical 'cabal shell' could work similarly. (In fact, Nix could probably produce a saner (shell) environment by using GHC environments, so the two solutions could converge.) Consider the following hypothetical session: $ cabal new-build # Creates .cabal.ghc.environment.arch-os-version, which is # non-default $ cabal shell # sets GHC_ENVIRONMENT to # $PWD/.cabal.ghc.environment.arch-os-version and spawns shell [cabal]$ ghc-pkg list ... (we see our customized environment) [cabal]$ exit $ cabal shell --run 'ghc-pkg list' ... (same output as previous) $ cabal shell --environment /path/to/ghc.environment [cabal]$ ghc-pkg list ... etc. Basically, "cabal shell" would be a wrapper over "GHC_ENVIRONMENT=/path $SHELL", but more friendly and with room for expansion.

Bryan Richter wrote:
I use nix-shell for choosing specific ghc environments, and undoubtedly a theoretical 'cabal shell' could work similarly. (In fact, Nix could probably produce a saner (shell) environment by using GHC environments, so the two solutions could converge.)
Consider the following hypothetical session:
$ cabal new-build # Creates .cabal.ghc.environment.arch-os-version, which is # non-default
$ cabal shell # sets GHC_ENVIRONMENT to # $PWD/.cabal.ghc.environment.arch-os-version and spawns shell [...] $ cabal shell --run 'ghc-pkg list' ... (same output as previous)
Note that cabal-install already provides a 'cabal exec' command which offers essentially that functionality for sandboxes, though in a different bikeshed color: $ cabal exec -- $SHELL $ cabal exec -- ghc-pkg list Cheers, Bertram

On 05/14/2018 03:50 PM, Bertram Felgenhauer via Haskell-Cafe wrote:
Bryan Richter wrote:
I use nix-shell for choosing specific ghc environments, and undoubtedly a theoretical 'cabal shell' could work similarly. (In fact, Nix could probably produce a saner (shell) environment by using GHC environments, so the two solutions could converge.)
Consider the following hypothetical session:
$ cabal new-build # Creates .cabal.ghc.environment.arch-os-version, which is # non-default
$ cabal shell # sets GHC_ENVIRONMENT to # $PWD/.cabal.ghc.environment.arch-os-version and spawns shell [...] $ cabal shell --run 'ghc-pkg list' ... (same output as previous)
Note that cabal-install already provides a 'cabal exec' command which offers essentially that functionality for sandboxes, though in a different bikeshed color:
$ cabal exec -- $SHELL
$ cabal exec -- ghc-pkg list
Heh, oops, I knew that. :) I have just been in the Nix swamps for too long recently. That said, this seems like the right solution. Rather than create used-by-default GHC environments, Cabal should modify "exec" to pick up a Cabal-generated environment.
participants (8)
-
Bertram Felgenhauer
-
Brandon Allbery
-
Bryan Richter
-
Carter Schonwald
-
Dan Burton
-
Gershom B
-
Herbert Valerio Riedel
-
Wolfgang Jeltsch