
Hi there -- I've groveled through the Hackage site, I've read a bunch of cabal documentation, and I've read far more blog entries, mailing list posts, and other things turned up by Google than I can count. Please forgive me if my confusions have been addressed elsewhere; if they have, I would appreciate a pointer. I'm having trouble wrapping my head around how to work with Hackage in the context of a Linux distribution that already has its own package management situation. Between distribution-provided packages containing Haskell programs and/or libraries, system-wide cabal installs, user-specific cabal installs, and cabal sandboxes, I can't seem to work out how the hackage/cabal architects intend people (ie, me) to juggle things. Here's a potential scenario: I want to use pandoc, for which my distro provides packages, so I install it along with its various and sundry dependencies using the package manager. Then I want to use another program for which my distro *doesn't* provide a package, so I use cabal-install to install it along with its various and sundry dependencies in ~/.cabal. Then I have a crazy idea I want to play around with in code. Initially, I've got all the libraries I need already installed, but eventually I discover there's a feature in foo-1.1.3.4 that my idea just can't live without. Unfortunately, foo-1.0.5.12 is required by pandoc. At this point, it would seem like cabalizing my little idea and letting it play by itself in a cabal sandbox is the preferred method. This seems reasonable, I say to myself, and proceed to recompile every single library upon which my little idea (recursively) depends. But then pandoc gets updated, as well as many of the libraries upon which it depends, which results in a wholesale upgrade of most of the system-wide libraries and now the stuff I've installed in my home directory has all manner of broken dependencies. There's probably a cabal-install command I can run at this point to clean things up (though presumably I'd have to run it for every source tree that uses a library that has been upgraded) but that's beside the point. My confusion is how this scales. Enough little ideas that depend on non-trivial libraries and I'm going to have eleventy-billion extremely similar copies of Data.Text littered about my home directory. Is this really the intention? Did I miss something? Are there better ``best practices'' than what I've described? Any guidance is most appreciated! Thank you. pete PS. Bonus question, which seems intimately related: if A depends on B and a new version of B appears, why does A necessarily need to be recompiled? Isn't one of the big wins of shared objects avoiding this very necessity? Presumably this is a consequence of how the GHC runtime does its linking, but I couldn't find helpful documentation on this count.

On Tue, Feb 4, 2014 at 10:29 PM,
I'm having trouble wrapping my head around how to work with Hackage in the context of a Linux distribution that already has its own package management situation. Between distribution-provided packages containing Haskell programs and/or libraries, system-wide cabal installs, user-specific cabal installs, and cabal sandboxes, I can't seem to work out how the hackage/cabal architects intend people (ie, me) to juggle things.
This is not a Hackage-specific issue; you will run into it with any language ecosystem, and every language ecosystem has its own mechanisms for dealing with it (usually involving some form of sandboxing). Indeed, some languages pretty much require those mechanisms: all too often, you need a separate rvm sandbox for multiple Ruby applications.... The general rule is that if you're just looking to install a few things available from OS packages and are not doing development, use the OS packages; otherwise, install just enough to be able to use cabal to install stuff. Although this also may vary: in many cases you will want to install the Platform libraries from your OS because you should really only have one version installed and it should generally be the "blessed" stable version. But if you're [say] intending to work with bleeding edge Yesod, then you probably want to only install ghc and cabal-install from the OS. Anyway, this question is why Haskell has hsenv, cabal-dev, and cabal sandboxes; Ruby has rvm, Python has virtualenv, and Perl has perlbrew, and other languages have their own mechanisms to deal with the same problems. And pkg-config is (de facto and for the moment) the solution C and C++ use, attacking it from a different direction. -- brandon s allbery kf8nh sine nomine associates allbery.b@gmail.com ballbery@sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net

Brandon Allbery wrote:
On Tue, Feb 4, 2014 at 10:29 PM,
wrote: I'm having trouble wrapping my head around how to work with Hackage in the context of a Linux distribution that already has its own package management situation. Between distribution-provided packages containing Haskell programs and/or libraries, system-wide cabal installs, user-specific cabal installs, and cabal sandboxes, I can't seem to work out how the hackage/cabal architects intend people (ie, me) to juggle things.
This is not a Hackage-specific issue; you will run into it with any language ecosystem, and every language ecosystem has its own mechanisms for dealing with it (usually involving some form of sandboxing). Indeed, some languages pretty much require those mechanisms: all too often, you need a separate rvm sandbox for multiple Ruby applications....
Pretty much this. Perhaps Nix can offer a more pleasant solution: http://ocharles.org.uk/blog/posts/2014-02-04-how-i-develop-with-nixos.html Best regards, Heinrich Apfelmus -- http://apfelmus.nfshost.com

Thanks! pete On Tue, Feb 04, 2014 at 11:48:52PM -0500, Brandon Allbery wrote:
On Tue, Feb 4, 2014 at 10:29 PM,
wrote: I'm having trouble wrapping my head around how to work with Hackage in the context of a Linux distribution that already has its own package management situation. Between distribution-provided packages containing Haskell programs and/or libraries, system-wide cabal installs, user-specific cabal installs, and cabal sandboxes, I can't seem to work out how the hackage/cabal architects intend people (ie, me) to juggle things.
This is not a Hackage-specific issue; you will run into it with any language ecosystem, and every language ecosystem has its own mechanisms for dealing with it (usually involving some form of sandboxing). Indeed, some languages pretty much require those mechanisms: all too often, you need a separate rvm sandbox for multiple Ruby applications....
The general rule is that if you're just looking to install a few things available from OS packages and are not doing development, use the OS packages; otherwise, install just enough to be able to use cabal to install stuff. Although this also may vary: in many cases you will want to install the Platform libraries from your OS because you should really only have one version installed and it should generally be the "blessed" stable version. But if you're [say] intending to work with bleeding edge Yesod, then you probably want to only install ghc and cabal-install from the OS.
Anyway, this question is why Haskell has hsenv, cabal-dev, and cabal sandboxes; Ruby has rvm, Python has virtualenv, and Perl has perlbrew, and other languages have their own mechanisms to deal with the same problems. And pkg-config is (de facto and for the moment) the solution C and C++ use, attacking it from a different direction.
-- brandon s allbery kf8nh sine nomine associates allbery.b@gmail.com ballbery@sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net
_______________________________________________ Beginners mailing list Beginners@haskell.org http://www.haskell.org/mailman/listinfo/beginners

On 02/04/2014 10:29 PM, tam@hiddenrock.com wrote:
Hi there --
I'm having trouble wrapping my head around how to work with Hackage in the context of a Linux distribution that already has its own package management situation. Between distribution-provided packages containing Haskell programs and/or libraries, system-wide cabal installs, user-specific cabal installs, and cabal sandboxes, I can't seem to work out how the hackage/cabal architects intend people (ie, me) to juggle things.
The most robust approach is to find or create distro packages for everything you need. I'm biased, but I think Gentoo has it best here because its hackport utility makes it trivial to create ebuilds for anything on hackage. We also gladly accept new ebuilds into our Haskell overlay. Arch and Debian are probably tied for second, but I don't know enough to say. In any case, you should track down the Haskell community for your distro and ask them about the easiest way to make a distro package from hackage.

On 02/06/2014 12:59 AM, tam@hiddenrock.com wrote:
The most robust approach is to find or create distro packages for everything you need.
How does this handle the situation where I want both package A and package B installed, but they each want a different version of package C?
How would your distro handle it if those packages were written in C? This isn't too much of a problem in practice. With Haskell it seems worse because everything has conservative dependencies in a *.cabal file and Cabal will refuse to build the thing unless they're met. When we find a package that's too conservative, we mangle the dependencies on the fly and report it upstream. If there's a more serious conflict, sometimes both versions can be installed side-by-side, but in general we have to file a bug and wait just like if it was written in e.g. python. But I see that as a good thing: I've got some 300 haskell packages installed at the moment, and I know that they're all in a consistent state (and up to date). It's pretty nice when people complain about Cabal and you have no idea what they're talking about. I also do development on three different machines, and it's handy to have them running the same packages.

How does this handle the situation where I want both package A and package B installed, but they each want a different version of package C?
How would your distro handle it if those packages were written in C?
I'd say the C-leaning factions of the open-source community have largely accepted the convention that ABI/API changes that cause incompatibilities are accompanied by bumps in the major version number. There are, of course, exceptions, but my feeling is they are rare, generally well-publicized, and quickly fixed. To be honest, I don't even know if hackage authors tend to follow this convention because...
With Haskell it seems worse because everything has conservative dependencies in a *.cabal file and Cabal will refuse to build the thing unless they're met.
... Haskell seems worse to me in that if package foo is updated, every other package depending on foo (recursely) must then be rebuilt, even if they didn't change and even if the interface presented by foo didn't change. C programs do not suffer similarly and my limited experience with ``cabal hell'' would have been a non-event if this recompilation process wasn't necessary. I don't know what technical issues necessitate it and I don't know whether the conventions required to make it feasible are already in place, which is partially what I'm trying to learn here. =) Additionally, it seems that the hackage ecosystem is far more intertwined than that of third-party Python packages (I have no experience with Ruby). That is, many things in hackage depend on other things in hackage whereas far more Python packages depend only on the libraries included in the base install. This is not a knock against Haskell by any means; it is, in fact, a testament to the powerful abstractions it supplies and enables. It does, however, weaken the ``other languages have this problem, too'' argument. Perhaps the solution is a more rigorous effort to keep the Haskell Platform up to date, especially with well-regarded ``building block'' libraries (eg, lens), and possibly even on a fixed schedule. Then hackage authors can be encouraged to make their stuff compatible with particular releases of the Haskell Platform and ``normal'' users would only need to worry about upgrades whenever the Platform is updated. But then we're really edging out on the slippery slope to Haskell becoming a distro unto itself. pete

On Thu, Feb 6, 2014 at 9:36 PM,
... Haskell seems worse to me in that if package foo is updated, every other package depending on foo (recursely) must then be rebuilt, even if they didn't change and even if the interface presented by foo didn't change.
This is something of a problem, yes... but it's not really "Haskell"'s fault. It's a specific design decision in GHC, in the name of performance: aggressive cross-module inlining means that specific compilation details of dependencies leak into the ABI of their dependents, necessitating exact ABI matches instead of simply consonant versions as in (normal) C. (For comparison, note that KDE has often had exactly the same problem in C++, through using cpp macros in place of inline functions for performance on "hot paths"; the exact macro and *every* (external or internal) function/method/variable it touches becomes part of the "public" ABI that must be maintained for compatibility.) Other Haskell compilers such do things differently. ajhc is a whole-program compiler, so in fact you *always* compile your dependencies again from source. uhc I don't think does the same amount of cross-module inlining, at a performance cost but maintainability gain. (And "Cabal hell" is shooting the messenger; it's just reporting the mess ghc has made / will make.) I do find myself wondering if anyone has checked to see how important the inlining is these days; perhaps ghc's general performance (and a hat tip to Moore's law) has made it possible to consider compiling more stable libraries. -- brandon s allbery kf8nh sine nomine associates allbery.b@gmail.com ballbery@sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net

Brandon Allbery wrote:
I do find myself wondering if anyone has checked to see how important the inlining is these days; perhaps ghc's general performance (and a hat tip to Moore's law) has made it possible to consider compiling more stable libraries.
Lately, I have been looking at GHC Core output for unrelated reasons, and it appears to me that cross-module inlining is essential to GHC performance. For instance, if you don't inline the monadic combinators (>>=) and `return` for the IO monad and perform even more inlining afterwards, performance can easily differ by an order of magnitude. The reason is that the second argument of (>>=) is a closure. Closure creation is very expensive compared to the sequence of primops you get after inlining. Best regards, Heinrich Apfelmus -- http://apfelmus.nfshost.com
participants (4)
-
Brandon Allbery
-
Heinrich Apfelmus
-
Michael Orlitzky
-
tam@hiddenrock.com