I'd also like to encourage the arch-haskell team to talk to Michael on this. The current Haskell repos have made things so much better than they were even a year ago, but if we can get behind a rising tide to lift all boats, I'd love to see that happen. I am still end user / novice enough to feel unqualified to evaluate Michael's proposals on a technical level other than they seem reasonable, maintainable and scalable as far as I can see. He's out in front on this, which is what Haskell needs right now. Let's get involved. -Ethan http://ethanschoonover.com/solarized On Wed, Dec 12, 2012 at 8:10 AM, Michael Snoyman wrote:

I definitely think there's a lot of room for collaboration here. I've been in touch with maintainers for other Linux distributions, and I think Stackage could become a project where different distros are all able to work together. Stackage is also set up in a way right now that seems to fit Arch's rolling release cycle.

To start off with, I would recommend using Stackage as a way to get a list of packages and their versions which work together correctly (the generated build-plan.log). You could then use automate a process to make Arch versions of all those packages.

Stackage provides a library in addition to an executable, and it can be extended to provide support for whatever facilities you need. This would also allow you to add in packages which are not currently part of the "official" Stackage set of packages. By using Stackage, you'll get a few benefits:

* A tool which is well tested for confirming that packages work well together. * The ability to work with other developers outside of Arch to improve this tooling. * As a community resource, I think Stackage stands a better chance of getting necessary modifications merged upstream. I've so far had very positive responses from package maintainers I've interacted with; I don't think a requested change has yet taken over a week to get implemented.

All that said, I'm not intimately familiar with the Arch Haskell processes and what hurdles you're trying to overcome. If you have concrete objectives, let me know what they are, and I'd be happy to figure out with you if Stackage would help achieve them.

Michael

On Wed, Dec 12, 2012 at 5:55 AM, Ramana Kumar wrote:

...

Dear Michael, Arch Haskell,

I saw this in the Haskell Weekly News recently: http://www.yesodweb.com/blog/2012/11/stable-vetted-hackage

I would like to propose that Arch Linux and the Hackage-packaging community project therein also be involved :)

Some information about the Arch Haskell project is here https://wiki.archlinux.org/index.php/Haskell_package_guidelines

I believe yesod and its dependencies, for example, are available in our [haskell-web] repository, and we (mainly Fabian and Magnus) are doing a great job of keeping it all working and up-to-date. How might this interface with stackage?

Do you (on either side) see potential for collaboration?

Cheers, Ramana

_______________________________________________ arch-haskell mailing list arch-haskell@haskell.org http://www.haskell.org/mailman/listinfo/arch-haskell

On Fri, Dec 14, 2012 at 10:18 AM, Magnus Therning wrote:

On Wed, Dec 12, 2012 at 4:55 AM, Ramana Kumar wrote:

...

Dear Michael, Arch Haskell,

I saw this in the Haskell Weekly News recently: http://www.yesodweb.com/blog/2012/11/stable-vetted-hackage

I would like to propose that Arch Linux and the Hackage-packaging community project therein also be involved :)

Some information about the Arch Haskell project is here https://wiki.archlinux.org/index.php/Haskell_package_guidelines

I believe yesod and its dependencies, for example, are available in our [haskell-web] repository, and we (mainly Fabian and Magnus) are doing a great job of keeping it all working and up-to-date. How might this interface with stackage?

Do you (on either side) see potential for collaboration?

This does indeed look very interesting. Would you, Ramana, have some time to have a closer look at what Michael is working on and see how that would fit with our goal of providing an Arch repo of bleeding edge Haskell packages? How does it affect our existing tooling, etc? (Note that, although it's been fun to write and use, I don't mind retiring 'cblrepo' for something better.) I won't really have time to look into it at the level of detail it deserves due to the workload at my employer at the moment.

I had a very cursory glance at what Michael has written about it so far and while it sounds interesting there a few issues that pop into my mind.

1. The set of packages is based on HP, in the past HP has been very slow to move on upgrading, in particular a few of the packages were left on yesteryears versions. This caused problems for us since non-moving packages tend to introduce requirements that limit how far other packages can be upgraded. Is it still the case that HP is this slow to move? How is the Stackage movement planning on countering this?

I think we're going to end up with a few different flavors of Stackage. Basing on the HP is good for typical end users who have installed the HP, but might not make much sense for your use case. I've added an option for Stackage to ignore the HP constraints, which is probably what you're going to want to do. (Alternatively, you *could* just stick with the HP constraints, as I don't think they're that onerous, but it's entirely your decision.)

2. Another issue is that of patching. I personally try to limit the patching, because it is a real time-suck, but sometimes it's not possible to avoid. In most cases I've opted to hold off on the upgrade until upstream have a chance to fix it. However, every such decision has ripple effects---non-upgradable packages have dependencies that prevent upgrading of other packages (same thing as with HP above). How will this be dealt with in Stackage, will old versions of packages be patched to allow upgrades of other packages in the set?

This will definitely be a feature I add to the Stackage codebase (it's currently an open issue[1] as I haven't thought through all the implications yet). But like you said, this can be a time suck. Additionally, the concept of having a distinction between Hackage's foo-1.2.3 and Stackage's foo-1.2.3 worries me a bit, but not enough to avoid putting the feature in. [1] https://github.com/fpco/stackage/issues/5

3. Is inclusion in Stackage a one-way road, can a package be removed? (This ties in with #2.) Upstream developers can sometimes be *very* slow with making changes that allow bumping dependencies. Resource limits on our side then prevent us from creating patches ourselves, which again result in preventing upgrades of other packages. It would be lovely to be able to temporarily remove a package from the set, especially if doing so has very few implications on other packages in the set. Other times packages simply become obsolete or unmaintained. This is something that's been discussed a few times, but we've never reached any sort of conclusion.

Packages can definitely be removed. There are really two categories of Stackage packages: explicit and implicit. Explicit packages are listed in Stackage.Config, and implicit ones are their deep dependencies. If an implicit package is no longer required by any explicit packages, it will be automatically dropped. We can also remove a package from the explicit list if the maintainer is unresponsive. So far, I've had great response times from maintainers. I'm hoping that by making this a centralized community effort, maintainers will be much more responsive than having individual Linux distros each making their own sets of requests. (Actually, IME so far maintainers have been happy to have a solid set of guidelines for improving their package quality, I'm hoping that trend continues.) The other tool we have in our disposal is temporary upper version bounds, e.g. [2]. Having a semi-fixed schedule of about a week to bump version bounds seems to work pretty well in practice. [2] https://github.com/fpco/stackage/issues/21

In short, after spending *very* little time looking at it I'm optimistic about Stackage. I think we might have to see how it fits our current goals, but then our goals are *not* set in stone, we've changed them before and can change them again if there are worthwhile benefits to doing so (tapping into a larger community is absolutely one large benefit).

I'm glad to hear it, and I'd really love to see Arch move forward with this. Let me know if you have any other questions, or if you can think of some tweaks/improvements to Stackage that would help you out. Michael