Fwd: Compatibility etiquette for apps, with cabal sandboxes and `stack`
On Sun, Nov 29, 2015 at 6:54 AM, Paolo Giarrusso
Hi all,
IIUC, people used to spend nontrivial effort making their Haskell tools work across a range of dependencies, and be careful about dropping support for older ones.
Given the tools we had, this was not easy. I was never aware of any tool that substantially helped with making sure that a package remained compatible with the range of packages that was theoretically allowed by the package's .cabal file.
Do cabal sandboxes or Stack reduce that need, at least for applications?* Or conversely, how bad is it to restrict support to users having them? I guess I am asking about common policies, but this probably depends on adoption of those tools.
IMO there is really no one right answer to this question. It depends on how nice you are, or whether someone is paying you. Obviously if someone is paying you, do as she says or do what she needs. If no one is paying you, then how nice do you want to be by expending the work? Honestly I'm not very nice. I keep stuff working in the current Stackage Nightly but that's it. Maintaining compatibility with huge dependency matrices is just an enormous amount of work. In my view, someone installing an application can just use stack and Stackage.
Omari Norman writes:
Someone installing an application can just use stack and Stackage.
I wonder how many people would be using XMonad, git-annex, etc. if this view were common place among application developers. I'm pretty sure that a large part of the user base of these tools has no clue "stack" exists, even, and the only reason why they can install these programs is because their distributions package manager allows then to do so without exposing them to any Haskell-specific build tools. Best regards, Peter
On Sunday, November 29, 2015 at 5:37:35 PM UTC+1, Peter Simons wrote:
Omari Norman writes:
Someone installing an application can just use stack and Stackage.
I wonder how many people would be using XMonad, git-annex, etc. if this view were common place among application developers.
I'm pretty sure that a large part of the user base of these tools has no clue "stack" exists, even, and the only reason why they can install these programs is because their distributions package manager allows then to do so without exposing them to any Haskell-specific build tools.
I guess you're thinking of different markets: "uses git-annex" is indeed (probably) orthogonal even to "knows there's a programming language called Haskell" (I might be exaggerating); I agree there distributing via Hackage is not a good option. But developers of, say, `hlint` have (I guess arguably) another target audience — Haskell developers; my question was about this audience, and Omari Norman's answer makes sense there.
Hi Paolo,
But developers of, say, `hlint` have (I guess arguably) another target audience — Haskell developers; my question was about this audience, and Omari Norman's answer makes sense there.
personally, I never install anything with stack or cabal-install, regardless of whether it's a tool intended for Haskell developers or not. Stack and cabal-install are great tools to use during code development, but neither of them has the capabilities I want from a package manager. Just my 2 cents, Peter
On Sun, Nov 29, 2015 at 11:37 AM, Peter Simons
Omari Norman writes:
Someone installing an application can just use stack and Stackage.
I wonder how many people would be using XMonad, git-annex, etc. if this view were common place among application developers.
Maybe zero. If an application developer cares about popularity, he should consider these things. Not all application developers care how many users they have. I'm pretty sure that a large part of the user base of these tools has no
clue "stack" exists, even, and the only reason why they can install these programs is because their distributions package manager allows then to do so without exposing them to any Haskell-specific build tools.
Distribution packagers are savvy enough to use stack. Furthermore, distributions do not install using cabal or from Hackage. Therefore, by your reasoning just as many people would be using XMonad, git-annex, etc. because the distribution packager would get the package, make the necessary alterations, and upload the distribution-specific package to the repository.
On 11/29/2015 01:37 PM, Omari Norman wrote:
Distribution packagers are savvy enough to use stack.
Ignoring the question of *how* that might work, most distributions forbid bundled dependencies because it creates a maintenance nightmare and fills our users' machines with untraceable security vulnerabilities. Literally no one does this, so I'm not sure what you're claiming here.
Furthermore, distributions do not install using cabal or from Hackage.
They do install from Hackage, just not using cabal-install.
Therefore, by your reasoning just as many people would be using XMonad, git-annex, etc. because the distribution packager would get the package, make the necessary alterations, and upload the distribution-specific package to the repository.
When using a real package manager, every package's dependencies must be satisfied simultaneously. Using stack isolates the developer from dependency conflicts with other packages during development, but when a user goes to install it, he doesn't have that luxury. If the developers of xmonad and git-annex both use stack/sandboxes, then it's possible that one of them will introduce a dependency that conflicts with the other, and neither developer will notice it thanks to the sandboxes. But if a user tries to install both at the same time, he can't, because (for example) xmonad wants foo-1.0 and git-annex wants foo-2.0. As a volunteer packager, I'm not going to fix that for you, I'm just going to work on something else whose upstream isn't a pain in the ass.
On Sun, Nov 29, 2015 at 2:12 PM, Michael Orlitzky
Furthermore, distributions do not install using cabal or from Hackage.
They do install from Hackage, just not using cabal-install.
So there's a distribution out there where end users pull source from Hackage, pull source for every dependency, and then build it all with GHC? If they're not doing what distributors like Debian does--building binaries--then what's the point of distributing at all? When using a real package manager, every package's dependencies must be
satisfied simultaneously.
True, but ouch, ultimately this is one factor that pushed me out of desktop Linux altogether. It's too hard to get packages for things I want to use, and then I'm fending for myself by building things. Centrally-planned packaging does not scale.
Using stack isolates the developer from dependency conflicts with other packages during development, but when a user goes to install it, he doesn't have that luxury.
He does if he uses stack. Grab a stack binary. It even installs GHC for the user.
Just thought of something that might help to deal with dependency nuisance: What if the top package (in namespace) contained major version? E.g. Cabal_1_22 below packages might use minor versions - basically, whenever api changes, change the version part of the affected package. Here by package I mean part of the namespace. This way, different versions could coexist quite painlessly.. ?
On 11/29/2015 02:39 PM, Omari Norman wrote:
So there's a distribution out there where end users pull source from Hackage, pull source for every dependency, and then build it all with GHC? If they're not doing what distributors like Debian does--building binaries--then what's the point of distributing at all?
Sure, all of the source-based distributions use the upstream tarball and compile it. The point of creating a "package" is so that you can have a real package manager manage your dependencies. Since most of the dependency info is contained in the cabal file, the packages are usually trivial. Gentoo, Nix, and FreeBSD all have tools that will convert a hackage package into a distribution package automatically.
When using a real package manager, every package's dependencies must be satisfied simultaneously.
True, but ouch, ultimately this is one factor that pushed me out of desktop Linux altogether. It's too hard to get packages for things I want to use, and then I'm fending for myself by building things. Centrally-planned packaging does not scale.
Given that almost all Linux systems in existence uses centrally-planned packaging, I don't believe that last claim. How many programs can you realistically keep installed and up-to-date with stack? Ten, twenty maybe -- if this is a serious hobby for you. One or two hundred if it's your full-time job? A typical Linux system will have hundreds of packages, and a system administrator will need to manage tens or hundreds of those systems. It's just not possible to do with something like stack -- you need one package manager that does everything. I'm not saying you need to rely on e.g. Debian upstream to create packages for you (I certainly don't), but you do need to have "system" packages for everything installed. This actually isn't very hard with those automated tools I mentioned earlier.
Using stack isolates the developer from dependency conflicts with other packages during development, but when a user goes to install it, he doesn't have that luxury.
He does if he uses stack. Grab a stack binary. It even installs GHC for the user.
If the user is highly technical and he wants to make stack administration his weekend activity. Otherwise, this isn't feasible for more than one or two packages. You can easily convince yourself of this: set up ten virtual machines, and install 20 packages using stack on each of them. Now keep them up-to-date for a year. If you're very good at bookkeeping and time management, it might even be possible. But it's not going to be a fun year. And 200 packages is barely enough to boot a single web server.
Am 29.11.2015 um 21:46 schrieb Michael Orlitzky:
On 11/29/2015 02:39 PM, Omari Norman wrote:
So there's a distribution out there where end users pull source from Hackage, pull source for every dependency, and then build it all with GHC? If they're not doing what distributors like Debian does--building binaries--then what's the point of distributing at all?
Sure, all of the source-based distributions use the upstream tarball and compile it. The point of creating a "package" is so that you can have a real package manager manage your dependencies. Since most of the dependency info is contained in the cabal file, the packages are usually trivial. Gentoo, Nix, and FreeBSD all have tools that will convert a hackage package into a distribution package automatically.
When using a real package manager, every package's dependencies must be satisfied simultaneously.
True, but ouch, ultimately this is one factor that pushed me out of desktop Linux altogether. It's too hard to get packages for things I want to use, and then I'm fending for myself by building things. Centrally-planned packaging does not scale.
Given that almost all Linux systems in existence uses centrally-planned packaging, I don't believe that last claim.
What does not scale is having to beg, bribe, or strong-arm upstreams into using a consistent set of library versions. You'll run into situation where application A wants libraries X.5 and Y.6, and application B wants X.6 and Y.4, and at that point, you'll have to make a hard decision between A and B. One way out is to make it so that multiple versions of the same library can be installed at the same time. C-based packages do this routinely by installing not libX and libY, but by installing libX-5, libX-6, libY.4, and libY.6. This still requires a mechanism to automatically select the right packaged lib, so the Haskell runtime will have to be told which libraries at what versions to combine with a given application. This could be totally easy or a huge PITA, I don't know enough about Haskell (I just happen to have a lot of administration experience with Linux). Another way out is to statically link each application, and avoid library packages entirely. It's viable only because we have multi-terabyte harddisks and multi-gigabyte RAM these days, and probably not what everybody wants to do. One thing that does not work at all in my experience is situations where you have a software ecosystem that's orthogonal to the OS. Typical package managers offer no way of having a local install, so my Eclipse installation typically consists of a download somewhere into my home directory, and plugins installed into that. Python is similar - I almost never install a Python application directly, I download it, use a virtualenv (Python's sandboxing method), and let it pull in and set up any dependencies it wants or needs. These local installs are all wheel reinventions, it would be better if apt, yum, rpm etc. supported local installs (in the form of "please install this into THAT directory inside my home dir, thank you very much"), and kept these installs separate. You can do stuff like that, but it requires expert knowledge so it's not an option for application installs.
How many programs can you realistically keep installed and up-to-date with stack? Ten, twenty maybe -- if this is a serious hobby for you. One or two hundred if it's your full-time job?
A typical Linux system will have hundreds of packages, and a system administrator will need to manage tens or hundreds of those systems. It's just not possible to do with something like stack -- you need one package manager that does everything.
True for operating systems. Or anything else that needs to "just work" without bothering about specific versions. Not so true for those individual applications. Of these, you often need a specific version, and since nothing in the OS depends on them, it's okay to have these installed independently of the package manager (but these applications can have such complicated dependency setups that they'll need their own package managers - Eclipse and Python come with such things for exactly that reason). Regards, Jo
On 29 November 2015 at 21:46, Michael Orlitzky
On 11/29/2015 02:39 PM, Omari Norman wrote:
So there's a distribution out there where end users pull source from Hackage, pull source for every dependency, and then build it all with GHC? If they're not doing what distributors like Debian does--building binaries--then what's the point of distributing at all?
Gentoo, Nix, and FreeBSD all have tools that will convert a hackage package into a distribution package automatically.
I'm not saying you need to rely on e.g. Debian upstream to create packages for you (I certainly don't), but you do need to have "system" packages for everything installed. This actually isn't very hard with those automated tools I mentioned earlier.
OK, that might be a (somewhat) reasonable alternative to using `cabal-install` or `stack` as a package manager — for users of those distros only. I know the mantra `cabal` is not a package manager, and that usually read as "`cabal` doesn't want to support its users' needs". I'm on OS X + Homebrew though. Also, as long as you just want to upgrade, a working `cabal upgrade` would be enough (and maybe within reach after the upcoming cabal changes); `stack upgrade-all` could also do the job. Bigger problems are removing packages and intermediate build products. -- Paolo G. Giarrusso - Ph.D. Student, Tübingen University http://ps.informatik.uni-tuebingen.de/team/giarrusso/
participants (6)
-
Imants Cekusins -
Joachim Durchholz -
Michael Orlitzky -
Omari Norman -
Paolo Giarrusso -
Peter Simons