In my view, a lack of communication like the one highlighted here is exactly what the Haskell Foundation hopes to improve. It's still being bootstrapped, so don't expect any action soon, but I would hope for forward motion by the springtime. Of course, the HF needs all of us to be the best foundation it can -- so please consider nominating yourself for a board position (https://haskell.foundation/board-nominations/) and/or applying for the executive director (full-time, salaried) position (https://haskell.foundation/ed-job-description/)! In either role, you could have direct impact on moving this all forward. Richard

On Dec 10, 2020, at 6:42 AM, Allen Sobot via hf-discuss wrote:

As a still (very) new Haskell user by many measures, indeed the little experience I had with Cabal isn't great overall in my view (honestly not a fan of Stack grabbing separate GHC instances though either unless I pass like 2 command-line flags, if there's a configuration option I'm not aware of I'd be grateful).

I had no clue about the communication issues etc. plaguing Cabal, that explains a bit...

In any case I'd like to try to contribute what I can to this effort, mainly to improve the UX which I agree is absolutely terrible in my humble opinion.

(Side note: I was informed on another mailing list that I may have some setting pertaining to forcing some kind of read receipt which is considered impolite on mailing lists, if this is still the case I apologize and I'm trying to determine what even is the precise issue or setting controlling it, at least on K-9 Mail and Neomutt)

On December 10, 2020 12:23:42 p.m. GMT+01:00, Ignat Insarov mailto:kindaro@gmail.com> wrote:

...

# Do something about Cabal?

Hello.

Cabal is the second most used tool in Haskell after GHC. It has many problems. It may be noticed that there is one and a half developers working on it. This is clearly not enough to address these problems. I propose that this is a good place to invest in.

### Problems I have in mind:

* Poor communication, lack of open source development process.

The whole Cabal–Stack schism appears to be an outcome of poor communication. One of the leading developers of Cabal is even banned from participation somewhere in Stack circles.[1] Personally, I reported several issues to Cabal and every single time it resulted in sadness. Observe a vicious circle: core developers are overworked ⇒ they are being unfriendly ⇒ there are fewer contributors ⇒ core developers are overworked.

I have no hard evidence but it appears that presently, more people that strive to improve the Haskell build experience are outside the Cabal cabal than are inside.

* User experience is an afterthought.

Cabal's user experience is horrifying. A collection of complaints is being compiled elsewhere.[2] There are also bugs being opened to Cabal because of this, requiring triage and therefore wasting the precious time of the few overworked developers. Stack is much more friendly — this shows by example that the user experience problem is not inherent and may be solved.

It is ordinary to receive output like this:

``` % cabal run example-executable Warning: The package list for 'hackage.haskell.org' is 84 days old. Run 'cabal update' to get the latest list of available packages. Resolving dependencies... cabal: Could not resolve dependencies: [__0] trying: example-0.1.0.6 (user goal) [__1] next goal: opaleye (dependency of example) [__1] rejecting: opaleye-0.7.1.0, opaleye-0.7.0.0 (constraint from project config TODO requires ==0.6.7006.1) [__1] rejecting: opaleye-0.6.7006.1 (conflict: example => opaleye^>=0.7) [__1] skipping: opaleye-0.6.7006.0, opaleye-0.6.7005.0, opaleye-0.6.7004.2, opaleye-0.6.7004.1, opaleye-0.6.7004.0, opaleye-0.6.7003.1, opaleye-0.6.7003.0, opaleye-0.6.1.0, opaleye-0.6.0.0, opaleye-0.5.4.0, opaleye-0.5.3.1, opaleye-0.5.3.0, opaleye-0.5.2.2, opaleye-0.5.2.0, opaleye-0.5.1.1, opaleye-0.5.1.0, opaleye-0.5.0.0, opaleye-0.4.2.0, opaleye-0.4.1.0, opaleye-0.4.0.0, opaleye-0.3.1.2, opaleye-0.3.1, opaleye-0.3, opaleye-0.2, opaleye-0.6.7002.0, opaleye-0.6.7001.0, opaleye-0.6.7000.0, opaleye-0.5.2.1, opaleye-0.3.1.1 (has the same characteristics that caused the previous version to fail: excluded by constraint '^>=0.7' from example) [__1] fail (backjumping, conflict set: example, opaleye) After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: opaleye, example ```

There are so many things that are wrong here. Even a sneaky _«to do»_ remark. If you wonder, in this case the solution is to remove and re-generate `cabal.project.freeze`.

Even the name of the program — it is actually _«cabal-install»_ — is incomprehensible, it should be re-branded to Cabal, which is how everyone calls it anyway.

* Features are not being introduced.

There is no reason for two build tools to exist. The killer feature of Stack — snapshots — should be supported by Cabal. Possibly Cabal itself should be refactored and split so that there are separate tools for packaging, version resolution and human interaction — I do not know. But certainly the way things are presently is a waste of developer effort and a source of confusion for everyone.

### My proposition, in particular.

* Ask all the people that show compassion to the cause of a great Haskell build tool to unite and work together on a better Cabal. This includes the developers of Stack and everyone that expressed unhappiness with the current state of Cabal. These people should be seen as a blessing, not as an obstacle. * Put in place a clear process for contributing and decision making, so that it does not come down to the privileged opinion of one of the core developers. * Make a model of user experience that Cabal should conform to, and make conformance a priority. Surely there are among us people that know a thing or two about user experience — call for them to step forward. Every issue that stems from misunderstanding, re-assign to the model instead of closing. * Merge the support of Stackage snapshots into Cabal. Ask the core developers of Stack to join the effort. Transition from Stack to Cabal should be one well discoverable command that just works.

I realize that this letter is largely an opinion piece. You can also see it as an _«ideal piece»_. Without an ideal, without a vision, we are stuck with the present. I do not insist that my vision is the best. But the present reality is not the best vision either. I propose, foremost, that we work and fight for a better future.

[1]: https://github.com/commercialhaskell/stackage/issues/4472 [2]: https://github.com/tomjaguarpaw/tilapia/issues?q=is%3Aissue+is%3Aopen+cabal _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

---

hf-discuss mailing list hf-discuss@haskell.org mailto:hf-discuss@haskell.org https://mail.haskell.org/cgi-bin/mailman/listinfo/hf-discuss https://mail.haskell.org/cgi-bin/mailman/listinfo/hf-discuss

Thanks Francesco. I have also been using Cabal since a long time ago. There is no question that some great things get done in Cabal. Mostly, Cabal does what it says on the box, and this is why I propose to improve it and not, say, move to Stack. But you may see that many people prefer the latter — this seems even more weird since, as you illuminate, Cabal can already interoperate with Stackage, so it is strictly more featureful. _(As far as I follow, Stack still has poor support for Backpack and sub-package build targets.)_ However, even in the light of the links you provide, we still cannot say that Cabal supports Stackage. You say:

...

There is no reason for two build tools to exist. The killer feature of Stack — snapshots — should be supported by Cabal.

As far as I know, this is already possible today! [1]

[1] https://github.com/fpco/stackage-server/issues/232 see also https://github.com/erikd/jenga/ https://hackage.haskell.org/package/stack2cabal

Heading to that link, the closing message says:

I've added a warning about the lack of support for revisions in cabal.config in f9632d7. Closing.

So, something is not working. Reading in more detail, there is evidently a disagreement between the core developers of Cabal and Stack. And as I understand, it has not been addressed ever since! This is exactly an example of the kind of communication problems that I alluded to in my first letter. Also, as far as I can see, there has been zero effort from the Cabal team to integrate these other tools that you point to.

Both tools have deficiencies, but looking at the "10 year overnight success" that is the recent step-change in the quality of Haskell's Language Server Protocol support, I am hopeful for the future. The answer is not as simple as "just use stack" or "just use cabal-install". I do not want to kick off a build-tool flamewar, but neither cabal-install nor stack are magical Christmas lands where everything is perfect. Here are a collection of objections to/my frustrations with stack, which is why "just use stack" isn't an answer to the problem: - Can't do GHCjs - Seems to like mass-rebuilding my work projects if I breathe on them the wrong way - Has its own way of doing everything that's slightly different (different command set, different ways of specifying build targets) - Fails to give me a REPL if it can't load all of a target's files first (I'm not sure exactly what's going on here, but when using dante, stack will blow up if the project contains a type error somewhere. Which is unhelpful when trying to get interactive editor support to resolve said type errors! Dante driving cabal does not have this problem. Extremely frustrating.) - Haskell programmers tend to end up coding to the lowest common subset of features across build tools. This held back adoption of backpack, which is sad. - hpack-by-default pushes another reinvention of basic haskell tooling, and does so with YAML, which is a file format that gets worse the more you understand it. (I expect its defenders will say this is just providing an option, but from my experience onboarding new Hackage uploaders, many people do not realise that hpack is optional.) - Many stack projects do not provide bounds on dependencies, relying on getting exact versions from LTS. This may be defensible for applications that sit at the leaves of a dependency graph, but less so for libraries. Accurate bound information is one of the reasons LTS build plans can be easily constructed, and tooling that encourages sloppy bounds feels to me like it encourages taking without giving back to the shared resource (the common set of libraries on Hackage with well-understood bounds). - What is casa.fpcomplete.com, and why was its downtime causing CI failures in my projects? What does stack talk to that domain about? HTH, -- Jack December 11, 2020 12:17 AM, "J Ho via hf-discuss" wrote:

What's wrong with Stack though? Works like a charm in all our projects, and with Nix integration takes care of the external c-libs etc dependencies quite nicely.

On Thu, Dec 10, 2020 at 3:07 PM Ignat Insarov wrote:

...

Thanks Francesco. I have also been using Cabal since a long time ago. There is no question that some great things get done in Cabal. Mostly, Cabal does what it says on the box, and this is why I propose to improve it and not, say, move to Stack. But you may see that many people prefer the latter — this seems even more weird since, as you illuminate, Cabal can already interoperate with Stackage, so it is strictly more featureful. _(As far as I follow, Stack still has poor support for Backpack and sub-package build targets.)_

However, even in the light of the links you provide, we still cannot say that Cabal supports Stackage. You say:

...

...

There is no reason for two build tools to exist. The killer feature of Stack — snapshots — should be supported by Cabal.

As far as I know, this is already possible today! [1]

[1] https://github.com/fpco/stackage-server/issues/232 see also https://github.com/erikd/jenga/ https://hackage.haskell.org/package/stack2cabal

Heading to that link, the closing message says:

...

I've added a warning about the lack of support for revisions in cabal.config in f9632d7. Closing.

So, something is not working. Reading in more detail, there is evidently a disagreement between the core developers of Cabal and Stack. And as I understand, it has not been addressed ever since! This is exactly an example of the kind of communication problems that I alluded to in my first letter. Also, as far as I can see, there has been zero effort from the Cabal team to integrate these other tools that you point to. _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

I found the whole cabal experience confusing and not well documented. I kept finding blogs online that were not working anymore in my version of cabal ('cabal sandbox init', 'cabal init --sandbox'...) when looking for advice. I still don't understand the whole v1-build, v2-build, new-build and build... The manual doesn't seem to have a decent conceptual overview of what the tool should do (e.g. an explanation of what sandboxing is supposed to achieve, or the 4 build command versions). Several of the links on this page https://www.haskell.org/cabal/ are dead https://www.haskell.org/cabal/release/cabal-latest/doc/API/Cabal/ or refer to old versions: https://wiki.haskell.org/Upgrading_packages This should not be construed as a critique of what has been achieved, but as honest feedback of my experience.

Innovation and advancing comes after competition, why shouldn't we embrace that wrt the tooling of Haskell? Duplication in efforts is a relative smaller problem IMHO. Well the competition has "go build" and cargo... And duplication of effort is a problem when there's not enough resources for even one decent build tool Immanuel

On Fri, Dec 11, 2020 at 01:54:57PM +0000, Tom Ellis wrote:

On Fri, Dec 11, 2020 at 10:31:23AM +0100, Immanuel Litzroth wrote:

...

I found the whole cabal experience confusing and not well documented. I kept finding blogs online that were not working anymore in my version of cabal

If you find outdated Haskell-related documentation please report it at

https://github.com/tomjaguarpaw/tilapia/issues/new

And on that point, I am tracking your particular issue with https://www.haskell.org/cabal/ as https://github.com/tomjaguarpaw/tilapia/issues/44 Tom

Wow, that's pretty nice! :) The main info that I think would be useful to add is how to contribute to this documentation, i.e. a link to the source repo. Am Mo., 3. Jan. 2022 um 13:43 Uhr schrieb Tom Ellis :

On Fri, Dec 11, 2020 at 03:09:38PM +0000, Tom Ellis wrote:

...

On Fri, Dec 11, 2020 at 01:54:57PM +0000, Tom Ellis wrote:

...

On Fri, Dec 11, 2020 at 10:31:23AM +0100, Immanuel Litzroth wrote:

...

I found the whole cabal experience confusing and not well documented. I kept finding blogs online that were not working anymore in my version of cabal

If you find outdated Haskell-related documentation please report it at

https://github.com/tomjaguarpaw/tilapia/issues/new

And on that point, I am tracking your particular issue with https://www.haskell.org/cabal/ as

https://github.com/tomjaguarpaw/tilapia/issues/44

I have subsequently fixed most of the outdated Cabal documentation on http://www.haskell.org/cabal. For more details see

* https://github.com/tomjaguarpaw/tilapia/issues/44

* https://github.com/tomjaguarpaw/tilapia/issues/47

Tom

_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

The correct URL to the up-to-date cabal user guide, since a couple of years ago or more, is: cabal.readthedocs.io In principle, someone could and should make a copy at www.haskell.org/cabal, but I don't know why it hasn't happened. Would be nice. cabal sandboxing is being phased out. Every blog on cabal sandboxing is out of date by definition. Don't bother. If your version of cabal-install is >= 3.0, when you issue "build" "run" etc they already mean v2-build, v2-run, etc. I refer you to the correct up-to-date cabal user guide, chapter 5 "nix-style local builds", for its model. I would not recommend spending time on the v1 model, unless you're a historian. "new-" was a transitional alias to "v2-" during a past transitional period that can be safely ignored (unless you're a historian). (Historians will remind me that the plan was, one day in the future, when the cabal devs introduce a 3rd model, the "new-" prefix will be revived again, this time aliasing to the 3rd model. I say that people are so happy with the current 2nd model that it won't happen.) On 2020-12-11 4:31 a.m., Immanuel Litzroth wrote:

I found the whole cabal experience confusing and not well documented. I kept finding blogs online that were not working anymore in my version of cabal ('cabal sandbox init', 'cabal init --sandbox'...) when looking for advice. I still don't understand the whole v1-build, v2-build, new-build and build... The manual doesn't seem to have a decent conceptual overview of what the tool should do (e.g. an explanation of what sandboxing is supposed to achieve, or the 4 build command versions). Several of the links on this page https://www.haskell.org/cabal/ are dead https://www.haskell.org/cabal/release/cabal-latest/doc/API/Cabal/ or refer to old versions: https://wiki.haskell.org/Upgrading_packages

This should not be construed as a critique of what has been achieved, but as honest feedback of my experience.

...

Innovation and advancing comes after competition, why shouldn't we embrace that wrt the tooling of Haskell? Duplication in efforts is a relative smaller problem IMHO. Well the competition has "go build" and cargo... And duplication of effort is a problem when there's not enough resources for even one decent build tool Immanuel

---

Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

On 10/12/2020 22.07, Jack Kelly via Haskell-Cafe wrote:

Both tools have deficiencies, but looking at the "10 year overnight success" that is the recent step-change in the quality of Haskell's Language Server Protocol support, I am hopeful for the future. The answer is not as simple as "just use stack" or "just use cabal-install". I do not want to kick off a build-tool flamewar, but neither cabal-install nor stack are magical Christmas lands where everything is perfect. Here are a collection of objections to/my frustrations with stack, which is why "just use stack" isn't an answer to the problem:

[--snip--] Absolutely agreed that there is currently to "Right Answer"(TM) wrt. Cabal or Stack. They do different things well/badly. Until we get a tool that can do everything the schism will probably remain. (And that's not even talking about issues like technical debt in each of the code bases, etc.) Anyway, just wanted to comment on a couple of points:

- hpack-by-default pushes another reinvention of basic haskell tooling, and does so with YAML, which is a file format that gets worse the more you understand it. (I expect its defenders will say this is just providing an option, but from my experience onboarding new Hackage uploaders, many people do not realise that hpack is optional.)

My problem here is that Cabal hasn't provided some very useful the features that YAML does out of the box, namely the ability to define arbitrary snippets of package configuration and to refer to them from multiple places. Examples: If I maintain a cohesive set of 5 packages, I need to duplicate various bits and bobs of information in 5 different places: - author info, license info - ghc options (I like to use identical options for all my packages in a 'set') - dependency version ranges(!) in 5 different cabal files. This is tedious at best and outright wrong at worst (inconsistent version bounds). YAML lets me avoid that annoyance -- yes, using entities and entity refs is a hackish way to do it, but it *works*. In no way am I advocating for YAML specifically. IMO, it's absurdly complex to cater for the needs of <1% of its users. However, I do wish Cabal could use some standardized format with similar capabilites for avoiding duplication in a full *generic* way. My problem with the cabal file format is that is basically entirely ad-hoc, esp. wrt. what I can avoid repeating over and over. (Dhall could work here since you can at least do inclusions and reuse in a generic way. Plus it has fully defined semantics plus implementations in at least a couple of langauges, Haskell and Scala.)

- Many stack projects do not provide bounds on dependencies, relying on getting exact versions from LTS. This may be defensible for applications that sit at the leaves of a dependency graph, but less so for libraries. Accurate bound information is one of the reasons LTS build plans can be easily constructed, and tooling that encourages sloppy bounds feels to me like it encourages taking without giving back to the shared resource (the common set of libraries on Hackage with well-understood bounds).

I posit that at least a *part* of the problem here is tedium for maintainers (see above), but while we're at it: I'm not sure manually maintaining bounds is at all scalable nor particularly "correct" (for lower bounds, esp.) unless one is *super*-vigilant about what bits of a dependency are *actually* in use. FWIW, I do try to do this for my packages, but I'm very confident that all of my packages probably have misleading lower bounds. It seems to me that some automated tooling is needed here, e.g. to try building/testing with e.g. all deps at the lower bound, all deps at the highest bound, etc. Regards,

On 11/12/2020 10.56, Oliver Charles wrote:

On Fri, 11 Dec 2020, at 9:50 AM, Bardur Arantsson wrote:

...

(Dhall could work here since you can at least do inclusions and reuse in a generic way. Plus it has fully defined semantics plus implementations in at least a couple of langauges, Haskell and Scala.)

Psst https://hackage.haskell.org/package/dhall-to-cabal https://hackage.haskell.org/package/dhall-to-cabal :)

Yes, thank for pointing that out :). I think I actually tried it at one point, but it was way to immature at the time... must have slipped my mind since then. IIRC my major problem at the time I tried it, it was also *really* slow. I mean "minutes" rather than "seconds" slow. I think that may have been due to performance problems in the Haskell implementation of Dhall, so hopefully it's not an issue these days. (It might also have been PEBKAC on my part, it's all a bit fuzzy...)

I think the missing piece here is making a frontend that transparently turns Dhall to Cabal syntax and runs cabal-install, so you don't ha ve to keep regenerating files.

That would be nice, but for large-scale adoption I think it probably needs to actually be an *official* format and perhaps even be upstreamed so that it gets maintained along with the Cabal format. (To account for things that change semi-often like extension names, new cabal settings, and whatnot.) Plus to know that it will be maintained well into the future. Thanks for working on it, btw. Might give it a shop when I get tired of hpack/YAML :). Regards,

Hi! I have not done any serious programming in Haskell (although at some point I would love to 😄) , I am interested in this topic, because this is one of the first obstacles of using it... I wonder if it's possible to appropriate GNU Guix as the distribution channel of many libraries. I heard it is now an experimental package in Debian😄 https://packages.debian.org/experimental/guix (FYI - After experimenting with both NixOS and Guix recently and I decided to favor Guix - but I notice Stack already has Integration with Nix?) And here is the Guix packaging guide I found for Python https://guix.gnu.org/manual/en/html_node/Python-Modules.html Cheers, Yasu

On Dec 11, 2020, at 18:57, Oliver Charles wrote:



...

On Fri, 11 Dec 2020, at 9:50 AM, Bardur Arantsson wrote: (Dhall could work here since you can at least do inclusions and reuse in a generic way. Plus it has fully defined semantics plus implementations in at least a couple of langauges, Haskell and Scala.)

Psst https://hackage.haskell.org/package/dhall-to-cabal :)

I think the missing piece here is making a frontend that transparently turns Dhall to Cabal syntax and runs cabal-install, so you don't have to keep regenerating files.

Ollie _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

I always wondered: do we need a package management system at all? Putting aside for a second the problem of interfacing with C or other external dependencies. What about a system where every source file is automatically annotated with a unique hash and all the required LANGUAGE extensions. For example: -- State explicitly any non-standard language extension {-# LANGUAGE MultiParamTypeClasses ,PatternSynonyms ,RankNTypes #-} -- Add to the module name its hash (calculated on its source contents, ignoring comments) module App.Main.K306f1981b41c -- Import a module, specifying the exact version import Data.Maybe.K306f1981823d .... main: ... Finally, we publish it on a shared content addressable system (hackage like). This module is in fact now a stand-alone mini-package that can be easily built as it states all its dependencies in a non-ambiguous way: lookMaNoPackages run App.Main.K306f1981b41c The nice thing is that it could be done in a completely backwards-compatible way, as it doesn't require any change to the language itself. And a modern tool like haskell-language-server could automate the hashing/annotations making it convenient to use. Would this work? titto Il giorno ven 11 dic 2020 alle ore 12:40 Jack Kelly via Haskell-Cafe < haskell-cafe@haskell.org> ha scritto:

December 11, 2020 7:50 PM, "Bardur Arantsson" wrote:

...

On 10/12/2020 22.07, Jack Kelly via Haskell-Cafe wrote:

...

[snip]

Absolutely agreed that there is currently to "Right Answer"(TM) wrt. Cabal or Stack. They do different things well/badly. Until we get a tool that can do everything the schism will probably remain.

(And that's not even talking about issues like technical debt in each of the code bases, etc.)

I'm glad my message was taken in the right way.

...

My problem here is that Cabal hasn't provided some very useful the features that YAML does out of the box, namely the ability to define arbitrary snippets of package configuration and to refer to them from multiple places. Examples:

If I maintain a cohesive set of 5 packages, I need to duplicate various bits and bobs of information in 5 different places:

[snip things that really should be DRY'd up by a modern build tool]

While it does not solve the problem you have in full, common stanzas can at least help you de-duplicate ghc-options settings within a package. https://cabal.readthedocs.io/en/latest/cabal-package.html?#common-stanzas

A possible solution: allow cabal.project to set common entries across multiple packages, and have `cabal sdist` write them into the cabal file that goes into the tarball? This is just me spitballing, so there are probably problems that I haven't considered.

...

This is tedious at best and outright wrong at worst (inconsistent version bounds). In no way am I advocating for YAML specifically. (Dhall could work here since you can at least do inclusions and reuse in a generic way. Plus it has fully defined semantics plus implementations in at least a couple of langauges, Haskell and Scala.)

I see that Oliver has already mentioned dhall-to-cabal . I'm a fan, but I'd be worried about bootstrappability if it became the main language of the main build tool.

...

...

- Many stack projects do not provide bounds on dependencies I posit that at least a *part* of the problem here is tedium for maintainers (see above), but while we're at it: I'm not sure manually maintaining bounds is at all scalable nor particularly "correct" (for lower bounds, esp.) unless one is *super*-vigilant about what bits of a dependency are *actually* in use.

FWIW, I do try to do this for my packages, but I'm very confident that all of my packages probably have misleading lower bounds.

It seems to me that some automated tooling is needed here, e.g. to try building/testing with e.g. all deps at the lower bound, all deps at the highest bound, etc.

No doubt. The only way to have a hope of doing this is with a CI matrix. https://github.com/haskell-CI/haskell-ci used to get some of the way there, but Travis CI recently yanked the rug out from under FOSS projects. I believe there is active work to make that repo support other CI systems. (More hands would probably help.)

A possible compromise for bounds: use ^>= bounds, raising the lower bound when you either a) need something from a new version, or b) move to a new major version? If you remain open to metadata revisions that relax the lower bound if needed, that might be OK? Again, spitballing, and I expect some kind soul to point out the flaws in this plan.

Best,

-- Jack _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

-- Pasqualino "Titto" Assini http://networkpolitics.svbtle.com http://quid2.org/

I don't actually see the problem in producing a hash of the API you use. It is simply the hash of the list of your imports. Is this overspecific? In the sense that it is more restrictive than it could be? Yes, but at least you get reproducibility and a permanently working build. Semantic versioning sounds good in theory and often works in practice, but you are relying on the goodwill and the precision of the library maintainer when both of these qualities are in short supply. The advantage of using module-level hashes is that you: * Avoid mistakes due to incorrect semantic versioning * Get maximum flexibility, as you are exploding the 'package cage' and accessing every file individually * Avoid a lot of 'false incompatibilities', when for example you would like to access a new version of a module in an upgraded package but you can't upgrade because you depend on another package that declares to be compatible only with the previous version of the package you want to upgrade to even when it's not actually using the new module you are interested in at all. The inconvenience of hashes is that they are cumbersome to manage by hand, but this can be automated. In fact, the question is not really semantic versioning vs hash-versioning. Both can be used at the module level as well as to the package level. The question is if we can simplify the dependency management problem by "going smaller", moving from a bunch of related modules (a package) as a unit of code exchange to a single module. In principle one could even go smaller, considering every function or data type [1] as an independently sharable entity. So, what is the optimal 'size' for code sharing: package, module, or entity? Best, titto [1] This actually make a lot of sense if you are interested in cross-language interoperability, see https://github.com/Quid2/zm for an example. Il giorno ven 11 dic 2020 alle ore 18:02 Joachim Durchholz ha scritto:

Am 11.12.20 um 13:48 schrieb Pasqualino "Titto" Assini:

...

I always wondered: do we need a package management system at all?

The main reason package management became a thing is indirect dependencies.

Without package management: Pick up a dependency, manually hunt down everything that it depends on, then get the dependencies' dependencies, and you never know how many are there to come. Lather, rinse, repeat with every update. Sometimes you'll need a completely new set of indirect dependencies after an update.

With package management, you specify the package you need and the package manager will do the rest. It should also tell you what it pulled in. If it's worth anything, you can tell it to use different dependencies; it's not a silver bullet that will solve your dependency incompatibilities for you, but it will give you a reasonable first approximation.

...

What about a system where every source file is automatically annotated with a unique hash and all the required LANGUAGE extensions.

Hashes are... difficult.

What you really want is a hash of the API you're using, possibly including nonfunctional properties. Problem is that it's really hard to even define the section of the API that you need - probably an unsolved problem, actually.

That's why people find semantic versioning so important. Usually, you have some global namespace where legal entities reside (Java did something that turned out to be pretty smart: domain names are per-entity anyway), with each entity assigning module names within their namespace. E.g. the owner of com.company manages com.company.module1, com.company.module2, etc. A module is identified by (sub)domain name and version number, throw in some recommendations about how to apply semantic versioning, and people will stop using those modules that don't have a good versioning policy anyway... and you're done.

It's not perfect, but it works.

(Hashes are overspecific, and they don't help with the relationship between version 1.2 and 1.2.2 of a module.)

(Disclaimer: This is from the perspective of the Java ecosystem, slightly generalized. Semantic versioning may be less of an issue for functional languages than it has been and will be for imperative ones, so I don't expect this to carry over 1:1.) _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

-- Pasqualino "Titto" Assini http://networkpolitics.svbtle.com http://quid2.org/

Il giorno sab 12 dic 2020 alle ore 13:20 Henning Thielemann < lemming@henning-thielemann.de> ha scritto:

On Sat, 12 Dec 2020, Pasqualino "Titto" Assini wrote:

...

I don't actually see the problem in producing a hash of the API you use. It is simply the hash of the list of your imports.

Is this overspecific? In the sense that it is more restrictive than it could be?

How do you address with a hash that added identifiers in an imported module do not harm if you import qualified or explicitly?

How do you address changed type of an imported variable?

The hash is obviously calculated on a normalised version of the module. As part of this normalisation step, all references to external definitions are fully qualified. And it is impossible to import a variable with a changed type, because if the type had changed so would have its definition and therefore the hash of the imported module.

Am 11.12.20 um 10:50 schrieb Bardur Arantsson:

FWIW, I do try to do this for my packages, but I'm very confident that all of my packages probably have misleading lower bounds.

It seems to me that some automated tooling is needed here, e.g. to try building/testing with e.g. all deps at the lower bound, all deps at the highest bound, etc.

With indirect dependencies, you can run into combinatorial explosion problems (number of combinations is roughly exponential with dependency chain length, give or take a few orders of magnitude). Just something to keep in mind when discussing design alternatives. (I don't know what designs would even make sense, or what kinds of metadata would be available to help guide version selection.) Regards, Jo

On 12/11/20 10:52 AM, Bardur Arantsson wrote:

Indeed, it's a difficult problem with no obvious solution, but given the combinatorial explosion issue, it's probably better to (somehow) leave it up to a computer to explore that space rather than humans. :)

I'm reminded of the headline from this Onion article: https://www.theonion.com/no-way-to-prevent-this-says-only-nation-where-this-... To fix dependency hell, you need only not cause it in the first place. You have to standardize the language and a useful bit of code that you dub the "standard library." Then anyone writing serious software in your language has to be careful not to break API compatibility, and to use semantic/libtool versioning when doing so is unavoidable. That's an O(n = number of packages) algorithm for detecting breakage because instead of detecting it in every consumer, you're _indicating_ it once. This is what separates languages that a few people use to write software for themselves from the languages that a lot of people use to write software for others. If you don't cause yourself these problems in the first place, then you don't need a not-invented-here package manager, because you can pick up Nix or Gentoo prefix or Anaconda or whatever and simply plug it in. The same is true of build systems. On the other hand, if you want to do something that you shouldn't be doing in the first place (living in Antarctica, climbing Mt. Everest, using a language that breaks itself on Tuesdays, etc.) then specialized tools are often necessary to bring the level of difficulty down from "impossible" to merely "really stupid."

On 12/11/20 2:48 PM, Henning Thielemann wrote:

On Fri, 11 Dec 2020, Michael Orlitzky wrote:

...

To fix dependency hell, you need only not cause it in the first place. You have to standardize the language and a useful bit of code that you dub the "standard library." Then anyone writing serious software in your language has to be careful not to break API compatibility, and to use semantic/libtool versioning when doing so is unavoidable. That's an O(n = number of packages) algorithm for detecting breakage because instead of detecting it in every consumer, you're _indicating_ it once.

Isn't this, what we are doing currently? When developing a library we try to stay within a major version range as long as possible, but if we need a breaking change, we increase the major version and have to adapt depending packages.

Some Haskell packages are pretty good in that regard, but my list was in order of importance. There is no such thing as "Haskell" nowadays, only "what GHC is doing this week," and the same goes for the simulacrum of a standard library that we call "base". The fact that, say, Cryptonite, is maintained intelligently is not as helpful when you consider that the words the author has written can change their meanings at any time.

Thanks, Ignat for bringing this issue! This is indeed a problem in the Haskell community and ecosystem at the moment, and we can't just ignore it if we want to improve things. Every Haskell developer uses the build tool. This is a crucial piece of development toolchain. That's why it's sad if a core tool is maintained only by a single person, maintainers are not willing to have more open-minded discussions regarding some user-requested features or a tool have poor UX. In my vision, everyone should be welcome to open issues to a build tool and contribute to it as everyone will benefit from improvements. And it's a pity that some people have a negative experience with this process which drives contributors away. I'm following the Cabal issue tracker, so I notice from time to time some not friendly behaviour towards its users. In the same spirit, having multiple build tools leads to duplication of effort for fixing the same problems and implementing the same features. Writing a build tool requires a colossal amount of effort and time. Not to mention that supporting both build tools is an enormous job as well. * If your project builds with Cabal, it's not guaranteed to build with Stack (at least you need to write stack.yaml). * If your project builds with Stack, it's not guaranteed to build with Cabal (you may need to specify upper and lower bounds). Thus, every maintainer must spend a lot of time maintaining support for both build tools if they want to provide good UX for everyone. I'm doing this for multiple years, and it's a troublesome process, but I do believe that thinking about users first is the way forward. This effort seems redundant in a theoretical world where Haskell has a single, core, official, open-sourced build tool managed by the community.

I have enjoyed using cabal very much

I'm also using Cabal for my personal projects, but this doesn't mean there are no problems. I learned to circumvent build tools pitfalls and survive in the ocean of incomprehensible errors, but I'm already in the middle of the ocean, and I don't want to stop swimming. Beginners, who are just staying on the coast, might not want to go into it at all. Best, Dmitrii On Thu, 10 Dec 2020 at 14:06, Ignat Insarov via hf-discuss < hf-discuss@haskell.org> wrote:

Thanks Francesco. I have also been using Cabal since a long time ago. There is no question that some great things get done in Cabal. Mostly, Cabal does what it says on the box, and this is why I propose to improve it and not, say, move to Stack. But you may see that many people prefer the latter — this seems even more weird since, as you illuminate, Cabal can already interoperate with Stackage, so it is strictly more featureful. _(As far as I follow, Stack still has poor support for Backpack and sub-package build targets.)_

However, even in the light of the links you provide, we still cannot say that Cabal supports Stackage. You say:

...

...

There is no reason for two build tools to exist. The killer feature of Stack — snapshots — should be supported by Cabal.

As far as I know, this is already possible today! [1]

[1] https://github.com/fpco/stackage-server/issues/232 see also https://github.com/erikd/jenga/ https://hackage.haskell.org/package/stack2cabal

Heading to that link, the closing message says:

...

I've added a warning about the lack of support for revisions in cabal.config in f9632d7. Closing.

So, something is not working. Reading in more detail, there is evidently a disagreement between the core developers of Cabal and Stack. And as I understand, it has not been addressed ever since! This is exactly an example of the kind of communication problems that I alluded to in my first letter. Also, as far as I can see, there has been zero effort from the Cabal team to integrate these other tools that you point to. _______________________________________________ hf-discuss mailing list hf-discuss@haskell.org https://mail.haskell.org/cgi-bin/mailman/listinfo/hf-discuss