I think Simon has a point. I really do not enjoy creating a new throwaway project for every small reproducer I want to test.

My (clearly faulty) mental model is this. - I have a particular stage2 compiler, say $(GHC). Maybe built in my build tree, maybe installed. - If I say `$(GHC) -package primitive`, it right says "I don't know about primitiver" - I expect to be able to say `cabal install primitive --with-compiler=$(GHC)`, to extend my $(GHC) with the package `primitive`. - Thereafter I expect `$(GHC) -package primitive` to work -- after all, I have extended GHC to know about it. If I'm in a cabal project then these installed-by-default packages shouldn't make any difference; cabal will control everything. I'm sure it used to be like this, but something has changed, and I'm not sure why. Clearly my model of cabal is faulty, but I don't know how to get an accurate one. Is there a writeup I can read? e.g. What is the "environments" file? Why does it fail to find Prelude? Would it be possible to support the simple story above, as well? Simon On Tue, 9 Jul 2024 at 13:23, Sebastian Graf wrote:

Hi Simon, Hi Matt,

I think Simon has a point. I really do not enjoy creating a new throwaway project for every small reproducer I want to test. A cabal project means that I can't simply pass `-fforce-recomp -ddump-simpl -O` to a GHC invocation, for example. Instead I have to create a cabal file and remember all the fields that need to be set. (Or <enter> myself through a perceived 30 step wizard with `cabal new`, only to find that I have to open the file anyway to tweak build-depends...)

Stupid as it may be, that subconsciously really affects my willingness to tackle an issue.

Presumably this can be solved through some amount of automation/scripting. Perhaps it would be enough to share these scripts in a prominent location, for example the Wiki. Alternatively, I think I would enjoy something like `nix-shell -p`, where I can just fire up a temporary shell with all the deps in a locally visible package db. That way I could keep calling `ghc` directly.

(Not that I suffer *much* from this issue at the moment anyway.)

Sebastian

Am Di., 9. Juli 2024 um 11:21 Uhr schrieb Matthew Pickering < matthewtpickering@gmail.com>:

...

`cabal install --lib` is very hard to use properly. I would advise against it.

It is much easier to create a simple cabal project and use normal cabal build commands. I packaged things in a repo for you.

https://github.com/mpickering/t25064

``` cabal build -w ghc-9.6.2 (fails) cabal build -w /path/to/devel/compiler (fails with error message) ```

Matt

On Tue, Jul 9, 2024 at 9:58 AM Simon Peyton Jones wrote:

...

Friends

I'm trying to repro #25064 with my development build of GHC. The test

case depends on packages `primitive` and `hashtables`. So I try this (see below).

...

Alas, apparently after the `cabal install`, it can't find Prelude.

What should I do? I tried removing the "environments" file, whatever

that is, which then meant it could find Prelude -- but the libraries were no longer installed.

...

I lack a decent model of what is going on with installing packages for

my development builds. Is there a write up anywhere?

...

Thanks

Simon

bash$ cabal install --lib hashtables primitive --with-compiler

$HOME/code/HEAD/$s2

...

Warning: Unknown/unsupported 'ghc' version detected (Cabal 3.10.1.0 supports 'ghc' version < 9.8): /home/simonpj/code/HEAD/_build/stage1/bin/ghc is version 9.11.20240517 Warning: Unknown/unsupported 'ghc' version detected (Cabal 3.10.1.0 supports 'ghc' version < 9.8): /home/simonpj/code/HEAD/_build/stage1/bin/ghc is version 9.11.20240517 Resolving dependencies... bash$ ~/code/HEAD/$s2 -c T25064.hs Loaded package environment from T25064.hs:1:8: error: [ ]8;; https://errors.haskell.org/messages/GHC-87110 \GHC-87110 ]8;; \] Could not load module ‘Prelude’. It is a member of the hidden package ‘base-4.20.0.0’. Use -v to see a list of the files searched for. | 1 | module Bug where | ^^^

bash$ rm /home/simonpj/.ghc/x86_64-linux-9.11.20240517/environments/default bash$ ~/code/HEAD/$s2 -c T25064.hs T25064.hs:3:1: error: [ ]8;; https://errors.haskell.org/messages/GHC-61948 \GHC-61948 ]8;; \] Could not find module ‘Control.Monad.Primitive’. Perhaps you meant Control.Monad.Writer (from mtl-2.3.1) Use -v to see a list of the files searched for. | 3 | import Control.Monad.Primitive | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

T25064.hs:4:1: error: [ ]8;; https://errors.haskell.org/messages/GHC-87110 \GHC-87110 ]8;; \] Could not find module ‘Data.HashTable.ST.Basic’. Use -v to see a list of the files searched for. | 4 | import qualified Data.HashTable.ST.Basic as H | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

---

ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

On 9.7.2024 16.35, Simon Peyton Jones wrote:

Would it be possible to support the simple story above, as well?

TL;DR the GHC developers (e.g. you, richard, sebastian) are virtually the only group of people who want to invoke GHC manually, from your POV the story might seem simple, but in grand scheme of things it's niche and obscure. `cabal-install` (and `stack` FWIW) don't want you to have any (implicit) state. That's not great for "real world projects". You may want to read through https://github.com/haskell/cabal/issues/6481 issue, which is kind of an opener towards support what you want. But again, it's niche feature. Creating a package as Matt proposes doesn't take much time (once you have done it few times), and allows to easily share the snippet (as everything can be contained inside package or/and project files). - Oleg

On Tue, Jul 09, 2024 at 01:06:02PM -0400, amindfv--- via ghc-devs wrote:

[...]I've talked to quite a lot of people who miss the old days of `cabal install`-ing a package (particularly if it requires tweaking flags) [...]

Forgot to mention an additional common workflow issue: having pre-installed packages that come from non-{H,St}ackage sources. Just another source of friction, where typing g-h-c-i-ENTER was by far the easiest way to use the package. Tom

On Tue, Jul 09, 2024 at 08:50:18PM +0300, Oleg Grenrus wrote:

cabal-install has various means to address that too. You can have `packages: /anywhere/onyourdisk` or `packages: http://somewhere.else/tar.gz in your cabal.project. Or you can have `source-repository-packages` for accessing remote git repositories. Or you can setup https://cabal.readthedocs.io/en/stable/config.html#local-no-index-repositori... for a bit more permanent and reusable solution.

cabal-install has ways to approach those problems *without having a stateful/implicit setup*. And the workflow is the same as usual workflow with packages and projects. In fact, real work projects do use source-repository-packages (and sometimes in project vendored `.tar.gz` source distributions).

Yep, I'm aware (and in fact have used `source-repository-package` on business-critical work projects. I appreciate its existence!). However, I'm specifically trying to get to the bottom of the sentiment that "the GHC developers (e.g. you, richard, sebastian) are virtually the only group of people who want to invoke GHC manually." My impression these past few years is that it's desirable to have one or more simple global states, accessible from e.g. ghci, and it's also desirable to avoid package conflicts, and there's tension between these needs. The above sentiment, though, seems to imply the former simply isn't desirable to cabal devs in the first place.

An anecdote from my side: I wrote`cabal-env` [1] for myself somewhat five years ago. I don't think I have used in the last two years. The package workflow is often better.

I've used it! My current work is another attempt to tackle a similar use-case. I disagree (strongly) that for my needs the package workflow is better. Cheers, Tom

On 10.7.2024 11.27, Simon Peyton Jones wrote:

I wonder if there is an articulate writeup of Cabal's mental model.

Sadly, i'm not aware of any. I'm also afraid, that Duncan's (who made initial v2-setup), Herbert's (who pushed v2 over the line) and mine models differ at least slightly. The https://well-typed.com/blog/2015/01/how-we-might-abolish-cabal-hell-part-2/ is Duncan's view from 2015. A side comment, we really should rename either the Cabal-the-library or cabal-install. Because I'm sure you are asking about mental model behind cabal-install v2-build, as "nothing" has changed in Cabal-the-library for a decade. The idea is that cabal-install v2-build installs everything into single global store, and gives consistent views to that store. In practice the global store is unusable directly, as a whole it may not be consistent nor unambiguous (see e.g. https://gitlab.haskell.org/ghc/ghc/-/issues/24823 for funny side-effects of that, you may have dozens of installed units in the package db with the same package name and version).

e.g.  What is an environment file?

Essentially it's a set of `-package-db` and `-package-id` flags. In other words, environment files are (specialized) response-files which ghc (tool, or lib) may pick up automatically. They are a way to encode a view into cabal's store (but technically there is no reason they cannot be used to give a view into stack's snapshot package databases too)

Why can't my GHC find Prelude? Probably because there is -hide-all-packages, -clear-package-db (or both); without -package-db and -package-id for base (a package with module Prelude).

If you invoke plain ghc, what packages does it "see"?

It depends on the state you have. This is complicated (use cabal-install). If you have a default or local environment file, ghc will use those. If there aren't it will use a global package db. To be honest, I'm not so sure what GHC does by default. I'm mostly familiar with programmatic execution, which often starts with `-clear-package-db`, `-hide-all-packages` etc to clear all the implicit stuff GHC may do. - Oleg

So far, the discussion around environment files in this thread has always been entangled with the idea of a "global state". GHC environment files (as written by `cabal install --lib`) are only global if they are placed in a directory where GHC will always look regardless of where GHC is currently running. By default, `cabal install --lib PKG` will place the file in such a location (~/.ghc/x86_64-linux-9.4.7/environments/default in my particular setup), but one can use the `--package-env DIR` flag to change the directory to use. GHC will look in the current directory and its parent directories for environment files, hence the following works: (choosing `some` as an arbitrary light-weight non-boot package; the `cabal install` command prints little because I already have a compiled copy in my global store) ~$ mkdir experiments ~$ cd experiments ~/experiments$ cabal install --package-env . --lib some Resolving dependencies... ~/experiments$ ghci Loaded package environment from /home/tom/experiments/.ghc.environment.x86_64-linux-9.4.7 GHCi, version 9.4.7: https://www.haskell.org/ghc/ :? for help Loaded GHCi configuration from /home/tom/git/dotfiles/.ghci Prelude> import Data.Some Prelude Data.Some> :q Leaving GHCi. ~/experiments$ cd .. ~$ rm -r experiments Of course, normal GHC (i.e. not GHCi) will also work and pick up the environment file. Using these "local" environment files allows one to create a local view of the global store that includes only a number of (presumably consistent) packages. Add to it by running `cabal install --package-env . --lib PKG` multiple times for different PKGs. No global state is modified, except potentially for compilation products of the libraries being installed -- but that is _append-only_ global state (the global cabal store), which is much less dangerous than general mutable global state. Furthermore, as already noted indirectly by Oleg, GHC environment files are human-readable, if not very easily human-writable. - Tom On 10/07/2024 11:57, Hécate via ghc-devs wrote:

Reading this certainly motivates me to push for a better design of the respective boundaries of GHC and cabal. Removing magic from GHC that it uses to compensate for the absence of cabal or other build system would certainly help using it for simpler use-cases.

Le 10/07/2024 à 11:53, Oleg Grenrus a écrit :

...

On 10.7.2024 11.27, Simon Peyton Jones wrote:

...

I wonder if there is an articulate writeup of Cabal's mental model.

Sadly, i'm not aware of any. I'm also afraid, that Duncan's (who made initial v2-setup), Herbert's (who pushed v2 over the line) and mine models differ at least slightly.

The https://well-typed.com/blog/2015/01/how-we-might-abolish-cabal-hell-part-2/ is Duncan's view from 2015.

A side comment, we really should rename either the Cabal-the-library or cabal-install. Because I'm sure you are asking about mental model behind cabal-install v2-build, as "nothing" has changed in Cabal-the-library for a decade.

The idea is that cabal-install v2-build installs everything into single global store, and gives consistent views to that store. In practice the global store is unusable directly, as a whole it may not be consistent nor unambiguous (see e.g. https://gitlab.haskell.org/ghc/ghc/-/issues/24823 for funny side-effects of that, you may have dozens of installed units in the package db with the same package name and version).

...

e.g.  What is an environment file?

Essentially it's a set of `-package-db` and `-package-id` flags. In other words, environment files are (specialized) response-files which ghc (tool, or lib) may pick up automatically. They are a way to encode a view into cabal's store (but technically there is no reason they cannot be used to give a view into stack's snapshot package databases too)

...

Why can't my GHC find Prelude? Probably because there is -hide-all-packages, -clear-package-db (or both); without -package-db and -package-id for base (a package with module Prelude).

...

If you invoke plain ghc, what packages does it "see"?

It depends on the state you have. This is complicated (use cabal-install). If you have a default or local environment file, ghc will use those. If there aren't it will use a global package db.

To be honest, I'm not so sure what GHC does by default. I'm mostly familiar with programmatic execution, which often starts with `-clear-package-db`, `-hide-all-packages` etc to clear all the implicit stuff GHC may do.

- Oleg

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

I agree and believe cabal v2-install has regressed from v1-install. My use case is I'm a hobbyist who writes small programs. I assume there are learners working outside a university class who are in a similar situation. Even experienced developers like Richard Eisenberg seem to agree, from https://github.com/haskell/cabal/issues/7832: Further, we do think the use case Richard has outlined of a "nice universe for a playground" is an important one. Great -- glad to hear it. Thank you. However, experience has repeatedly shown that "do what I mean" algos in package management lead to frustrated and confused users, and while things should be as easy as possible, they should also be no easier than that, and finding that balance is not going to be simple. I agree here, and have not thought about the specific of the algorithms above. But, there is an algorithm that is known to (mostly) work: what v1 did. That is, the universe expands only by user request, when that user asks to add a new package to their universe. At that point, cabal selects the most recent release of that package (or whatever version the user asked for). Then, cabal uses the same algorithm it usually uses to decide its dependencies. This worked for the lifetime of v1. But it only mostly worked, because of cabal-hell: that's why v2 exists! So there would be a potential pitfall in that users try to build an inconsistent universe, or they build a consistent universe that doesn't serve their actual needs. I'm not sure those pitfalls are avoidable... but maybe they could be mitigated with cabal destroy-universe which blows away the current universe, allowing the user to re-expand it from scratch. (In my own practice, a universe went stale at about the same rate as GHC released, and so I just moved to a new GHC when things got rough in the universe.)... IMHO the current situation is a deterrent to beginners trying to learn Haskell. There is a huge amount to learn and adding cabal and stack etc can appear as a huge pain to them, particularly if they have used pip3, the Python package manager. Also I'm surprised that Cabal isn't under the control of the Core Library committee, I think that would have made for a smoother transition from cabal v1 to v2 Thanks George On Tue, Jul 9, 2024 at 2:06 PM amindfv--- via ghc-devs wrote:

On Tue, Jul 09, 2024 at 05:07:07PM +0300, Oleg Grenrus wrote:

...

On 9.7.2024 16.35, Simon Peyton Jones wrote:

...

Would it be possible to support the simple story above, as well?

TL;DR the GHC developers (e.g. you, richard, sebastian) are virtually the only group of people who want to invoke GHC manually [...]

Genuinely curious, what makes you say this?

My only data is anecdata, but I've talked to quite a lot of people who miss the old days of `cabal install`-ing a package (particularly if it requires tweaking flags) and then having it at your fingertips whenever you run GHCi, `runghc`, or GHC. Some packages are so central to our workflows that it's almost like needing to `mkdir foo ; cd foo ; cabal init ; $EDITOR foo.cabal ; cabal repl` to get the Prelude.

Nobody misses the package conflicts that could come from long-term thoughtless default-global installation, but to say that almost nobody wants to invoke GHC manually seems like a major stretch. If we had a cleaner solution, I'm willing to bet a lot of people would drop the `cabal init` workflow for everyday tinkering like a hot potato.

Tom

P.S. I've built my own hacks on top of GHC environment files to recapture this way of working, but that's another story.

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

On 9.7.2024 15.23, Sebastian Graf wrote:

I can't simply pass `-fforce-recomp -ddump-simpl -O`

You can, `cabal build --ghc-options= "-fforce-recomp -ddump-simpl -O"` should work (though it applies to all packages).

and remember all the fields that need to be set. (

`cabal init` is your friend. That said, there are very little fields to be set for minimal package, `cabal-version, name, version` and the library pragma with `build-depends` and `exposed-modules`. After you have done such package few times, you will remember those easily. I'd argue a lot more easily that remembering how to use `cabal install --lib` properly. - Oleg

One other alternative is to use a cabal script. You can simply add a multiline comment to your haskell files: {- cabal: build-depends: base, primitive, hashtables -} Then you can run the file with ‘cabal run T25064.hs’, or you can add a shebang at the top of the file and make it executable: #!/usr/bin/env cabal For more info see the cabal docs: https://cabal.readthedocs.io/en/latest/cabal-commands.html#cabal-run Cheers, Jaro