On Thu, Jan 30, 2014 at 11:40 PM, Clark Gaebel wrote:

How does the process of taking over maintenance add latency to your work?

...

4) Email hackage admins for upload rights.

Emails are hard. There's a massive cost in the context switch from coding to writing a justification as to why someone other than maintainer gets upload rights. At least that's what I gleaned from Roman's email. -- Kim-Ee

I HAVE A SOLUTION for anyone who asks me, I will help manage the "this isn't maintained, i wanna take over maintership" pestering emails that need to happen. that said, you'll still need to email me to ask me to do that, or pester me on IRC as applicable. (which may take just as much time as "hey i'd like to take over maintainership of X on hackage please" sent to the libraries list) Point being, if someone finds the prospect of doing the pestering needed to do the process overwhelming, ask me nicely, and i'll try to help push it along (while keeping them in CC and such, though they of course will have to chime in at some point) should this thread also touch on the the libraries mailing list? ccing it just in case :) point being, we need to have a responsive, *responsible* way of quickly resolving these things that easy to do. time for comments: 2 weeks -Carter On Thu, Jan 30, 2014 at 11:53 AM, Ben Gamari wrote:

Clark Gaebel writes:

...

How does the process of taking over maintenance add latency to your work?

1) Check out broken version of package. 2) Fix locally, bump version number locally. 3) cabal sandbox add-source ../fixed-package in any package that needs the fixed version. 4) Email hackage admins for upload rights. 5) Continue working on your actual project. 6) Receive upload privileges one day. 7) Upload fixed package.

As far as I can tell, the only real latency cost here is that paid to fix the broken version.

In my experience, step 4 involved several round trips between a number of different people. Admittedly, this is in part because it's easy to forget about the broken package after you have fixed it locally.

Cheers,

- Ben

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

On Thu, 2014-01-30 at 13:11 -0500, Gershom Bazerman wrote:

(ccing hackage admin alias directly, just to make sure the right people are seeing all the emails)

I think there are two _different_ issues we've been conflating under "taking over a package".

1) A package is genuinely unmaintained and it should be taken over by someone else, and someone steps up. This is fine to take some time, in my book. Our current process, as described by Johan Tibell, is reasonable. It _should_ be easily accessible from the hackage homepage, _and_ it should be on the haskell wiki. Perhaps the homepage could link to a hackage FAQ on the wiki? Volunteers on any of this?

Right, Johan described our policy, and indeed that should be documented in obvious places.

2) A package needs patching to work with the latest deps, and the maintainer is unreachable, then we just want to upload a fixed version soon. But who knows, maybe we don't want to take it over for good. Maybe the maintainer is just on a monthlong vacation on a small island chain. Maybe they're sick, or busy at work. This is the sticking point.

So maybe we could have a different policy on the latter. There are people like Roman, Ben, others, who I feel like I sort of "know" from their presence in the community, whether or not I've met them personally. I'd feel comfortable letting trusted community members not "take over" packages entirely without process, but use a more relaxed process simply to upload simple patches for compatibility, etc.

Is there some way that we could create a "two process" approach -- one for taking over a package, and a more relaxed one for trusted individuals to fix things up easily?

So this one is easy so long as the maintainer (or one of the maintainers) is reachable, because then a maintainer can spend 30 seconds to add the person volunteering to make the fix to the maintainer group for the package. If the maintainer(s) is/are unreachable then that's a hard call. They have not pre-approved (which they can do by adding some trusted person to the maintainer group) and they are not in a position to approve or not in the short term as they are unreachable. In general we like to give authors/maintainers strong control over their packages. I should note that a maintainer could pre-approve some helper to whom they can delegate the decision. For example, suppose volunteered to be a person of good taste who would review cases where the package needs fixing in simple ways but the maintainer is away. That person does not have to be the one who develops the fix or even necessarily the person who uploads it. What the maintainer would need to do is to add that volunteer of good taste to the maintainer group, and then that volunteer now has all the same rights as the "real" maintainer and so can upload the fixed package themselves or even add the person who did the fix to the maintainer group too (different maintainers and fixers might agree different policies here). But otherwise, in general I think we do not want to force maintainers to accept "fixes" from other developers without any kind of pre-approval. So the worst case of there being no pre-approved helper and the maintainer uncontactable in the short term would simply remain. On the other hand we can certainly encourage and ease setting up pre-approval with volunteers, especially for important packages. Or just encourage people to add secondary/backup maintainers to their packages. A little bit of planning will avoid the angst. -- Duncan Coutts, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

On 30/01/14 18:47, Michael Orlitzky wrote:

On 01/30/2014 01:39 PM, Duncan Coutts wrote:

...

I should note that a maintainer could pre-approve some helper to whom they can delegate the decision. For example, suppose volunteered to be a person of good taste who would review cases where the package needs fixing in simple ways but the maintainer is away. That person does not have to be the one who develops the fix or even necessarily the person who uploads it. What the maintainer would need to do is to add that volunteer of good taste to the maintainer group, and then that volunteer now has all the same rights as the "real" maintainer and so can upload the fixed package themselves or even add the person who did the fix to the maintainer group too (different maintainers and fixers might agree different policies here).

I don't suppose there's any support for groups in hackage2? I don't know anyone in particular that I'd want to pre-delegate this responsibility to, but I would be fine with e.g. delegating it to "the hackage team" (who I already implicitly trust).

They could then make the decision on my behalf, using something like Gershom's "I feel like I can trust this guy" criteria.

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Is [1] what you're after? It was unused until recently as far as I know. [1]: https://hackage.haskell.org/packages/trustees/ -- Mateusz K.

As one of the hackage admins, I should probably weigh in here too. First, let's not forget that one of the big new features of the new hackage was the addition of proper permission checks for uploading new versions of packages. Previously, anyone could upload any package, which is a huge security risk for one, and also violates the expectations of package owners as well. So I think this feature is a good one. Second, there are a lot of abandoned packages. It is good for the community if people take over maintenance of these, so the process for doing this should be as smooth as possible. Note that these two things are at odds. So we have to find a balance. Johan already mentioned the process we use, although we recently discussed changing it a bit so that the hackage admins would email the maintainer instead of the user wanting to take over. This is because of a recent mishap where a package was taken over because the maintainer missed the initial email (and haskell-cafe message). I agree that this process is suboptimal, at least for some scenarios. It's a lot of work for both the user and the hackage admins. Also, it's unclear to me how long the wait periods have to be. The distinction between actually taking over maintainership and just doing some small fixes seems to be a good one, though it's not exactly clear where to draw the line. Fixing compilation issues on newer GHCs and relaxing dependencies seems ok to me, though. There is support for trustees on the new hackage, as other's mentioned. These can maintain any package. Perhaps this would be a good time to recruit a few of them? Regards, Erik On Thu, Jan 30, 2014 at 7:39 PM, Duncan Coutts wrote:

On Thu, 2014-01-30 at 13:11 -0500, Gershom Bazerman wrote:

...

(ccing hackage admin alias directly, just to make sure the right people are seeing all the emails)

I think there are two _different_ issues we've been conflating under "taking over a package".

1) A package is genuinely unmaintained and it should be taken over by someone else, and someone steps up. This is fine to take some time, in my book. Our current process, as described by Johan Tibell, is reasonable. It _should_ be easily accessible from the hackage homepage, _and_ it should be on the haskell wiki. Perhaps the homepage could link to a hackage FAQ on the wiki? Volunteers on any of this?

Right, Johan described our policy, and indeed that should be documented in obvious places.

...

2) A package needs patching to work with the latest deps, and the maintainer is unreachable, then we just want to upload a fixed version soon. But who knows, maybe we don't want to take it over for good. Maybe the maintainer is just on a monthlong vacation on a small island chain. Maybe they're sick, or busy at work. This is the sticking point.

So maybe we could have a different policy on the latter. There are people like Roman, Ben, others, who I feel like I sort of "know" from their presence in the community, whether or not I've met them personally. I'd feel comfortable letting trusted community members not "take over" packages entirely without process, but use a more relaxed process simply to upload simple patches for compatibility, etc.

Is there some way that we could create a "two process" approach -- one for taking over a package, and a more relaxed one for trusted individuals to fix things up easily?

So this one is easy so long as the maintainer (or one of the maintainers) is reachable, because then a maintainer can spend 30 seconds to add the person volunteering to make the fix to the maintainer group for the package.

If the maintainer(s) is/are unreachable then that's a hard call. They have not pre-approved (which they can do by adding some trusted person to the maintainer group) and they are not in a position to approve or not in the short term as they are unreachable. In general we like to give authors/maintainers strong control over their packages.

I should note that a maintainer could pre-approve some helper to whom they can delegate the decision. For example, suppose volunteered to be a person of good taste who would review cases where the package needs fixing in simple ways but the maintainer is away. That person does not have to be the one who develops the fix or even necessarily the person who uploads it. What the maintainer would need to do is to add that volunteer of good taste to the maintainer group, and then that volunteer now has all the same rights as the "real" maintainer and so can upload the fixed package themselves or even add the person who did the fix to the maintainer group too (different maintainers and fixers might agree different policies here).

But otherwise, in general I think we do not want to force maintainers to accept "fixes" from other developers without any kind of pre-approval. So the worst case of there being no pre-approved helper and the maintainer uncontactable in the short term would simply remain.

On the other hand we can certainly encourage and ease setting up pre-approval with volunteers, especially for important packages. Or just encourage people to add secondary/backup maintainers to their packages.

A little bit of planning will avoid the angst.

-- Duncan Coutts, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

On Thu, 2014-01-30 at 20:23 +0100, Erik Hesselink wrote:

There is support for trustees on the new hackage, as other's mentioned. These can maintain any package. Perhaps this would be a good time to recruit a few of them?

I should point out that trustees cannot upload new package tarballs. They can upload docs and will be able to tweak .cabal metadata (when that feature is fixed) but we drew the line at source code changes. -- Duncan Coutts, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

On Friday, January 31, 2014, Duncan Coutts wrote:

On Thu, 2014-01-30 at 20:23 +0100, Erik Hesselink wrote:

...

There is support for trustees on the new hackage, as other's mentioned. These can maintain any package. Perhaps this would be a good time to recruit a few of them?

I should point out that trustees cannot upload new package tarballs. They can upload docs and will be able to tweak .cabal metadata (when that feature is fixed) but we drew the line at source code changes.

True, but that can be changed if people think it's a good idea. It seems they don't, currently, but I have a patch in case they do ;) Erik

Because I can't release my package (which depends on the fixed version) immediately. Also, all sorts of psychological reasons (as Kim-Ee Yeoh said) — context switches, open loops etc. It's much better to just upload the package and be done with it. Roman * Clark Gaebel [2014-01-30 11:40:12-0500]

How does the process of taking over maintenance add latency to your work?

1) Check out broken version of package. 2) Fix locally, bump version number locally. 3) cabal sandbox add-source ../fixed-package in any package that needs the fixed version. 4) Email hackage admins for upload rights. 5) Continue working on your actual project. 6) Receive upload privileges one day. 7) Upload fixed package.

As far as I can tell, the only real latency cost here is that paid to fix the broken version.

Regards, - Clark

On Thu, Jan 30, 2014 at 11:30 AM, Roman Cheplyaka wrote:

...

In the recent past I took over two unmaintained packages: bert and ansi-terminal. I don't mind spending a bit of time to keep our ecosystem from bitrotting.

However, both times I had to go through an irritating procedure of contacting hackage admins, asking them to grant me upload rights, explaining why the maintainers can't do that themselves and why I think the packages are abandoned.

Instead of a feeling that I'm doing something good and useful, I have a feeling that I'm bothering people with my own problems. It also adds unnecessary latency to my work.

So from now on I'll simply fork the packages I need to fix.

Others are of course welcome to use my forks.

(This email was prompted by regex-tdfa which doesn't build on GHC 7.8, and whose maintainer hasn't responded. My fork is at http://hackage.haskell.org/package/regex-tdfa-rc .)

Roman

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

-- Clark.

Key ID : 0x78099922 Fingerprint: B292 493C 51AE F3AB D016 DD04 E5E3 C36F 5534 F907

* Ben Gamari [2014-01-30 11:49:37-0500]

Roman Cheplyaka writes:

...

In the recent past I took over two unmaintained packages: bert and ansi-terminal. I don't mind spending a bit of time to keep our ecosystem from bitrotting.

However, both times I had to go through an irritating procedure of contacting hackage admins, asking them to grant me upload rights, explaining why the maintainers can't do that themselves and why I think the packages are abandoned.

Instead of a feeling that I'm doing something good and useful, I have a feeling that I'm bothering people with my own problems. It also adds unnecessary latency to my work.

So from now on I'll simply fork the packages I need to fix.

Others are of course welcome to use my forks.

(This email was prompted by regex-tdfa which doesn't build on GHC 7.8, and whose maintainer hasn't responded. My fork is at http://hackage.haskell.org/package/regex-tdfa-rc .)

I have also tried to upstream GHC 7.8 fixes to this package to no avail. However, the maintainer did inform me that the darcs repository is here[1].

It would be great if someone (it could be me, Roman, or anyone else interested) would take over the regex-tdfa package itself. While the forking idea certainly eliminates a lot of effort for the fork-er, it would be nice if the original point in the namespace were inhabited by an actively maintained package. I feel like an explosion of forks gives a not-so-great impression of the Haskell community and makes it even harder for beginners to find the right packages to depend upon.

Concerning the friction imposed by the package take-over process, I agree that it is rather onerous. In my opinion the protocol is a bit too thorough for its own good.

Cheers,

- Ben

[1] http://code.haskell.org/regex-tdfa/

Here's my git repo (imported from that repository + my patches): https://github.com/feuerbach/regex-tdfa-rc If anyone eventually takes over, feel free to use it (the renaming commits are easy to strip). Roman

* Erik de Castro Lopo [2014-01-31 09:22:36+1100]

I really can understand why you did this; I am frustrated by some of the same issues. However, I think if any significant number of people did this, the results could easily be disasterous.

Agreed. Maybe we need those disasterous results to realize that the current process is bad and come up with a better one. Or maybe it's just me, and everyone else is happy (enough) with the process, so nothing will happen. Roman

So let's talk about security. The current process protects against malicious parties rather poorly. It protects against malicious takeover of a maintained package, but an adversary could presumably find an unmaintained or semi-maintained yet popular package (such as ansi-terminal) to pull off an attack. The process does protect against *targeted* malicious takeover (i.e. an adversary wants to inject malicious code in exactly the (maintained) package K, and not just any moderately popular (but unmaintained) package L). But that is not enough. So security and trust have to be enforced by a different mechanism anyway. Roman * Evan Cofsky [2014-01-31 06:11:52+0000]

Hello,

As you all know I'm new to Haskell, but not at all new to software communities or computer security. The Haskell community is maturing, and as part of that we will have to be able to ensure that social engineering attacks against our core infrastructure, such as the canonical package repository, are not trivial to pull off successfully.

There are several instances just in the past week that have been making the rounds of basic impersonation attacks having disastrous consequences because people were quite willing to trust complete strangers, and of course we all know about the various problems faced by the Node and the Ruby communities and their package management systems, which were fully automated.

I put a lot more faith into Haskell programs, probably naively so, but as people, though, we have to at least maintain a certain amount of healthy skepticism when it comes to responding to requests to take over projects, and without a doubt along with that skepticism comes friction and process. But these are absolutely essential to having a trustworthy resource in Hackage, and I think that far outweighs the ficticious freedoms offered by management by handshake and a smile, especially for such an important community resource.

-- Evan Cofsky

* Erik Hesselink [2014-01-31 12:45:56+0100]

Security is never binary, and just because we're not guarding against all scenarios, doesn't mean we shouldn't guard against any.

It doesn't. It means, however, that we should at least ask ourselves whether it's worth what we pay for it. And whether we can get better for less.

Again, do you have any suggestions to make things better?

Here I merely want people to realize that there is a problem. How to solve it is a whole new discussion.

On Fri, Jan 31, 2014 at 12:02 PM, Roman Cheplyaka wrote:

...

So let's talk about security.

The current process protects against malicious parties rather poorly. It protects against malicious takeover of a maintained package, but an adversary could presumably find an unmaintained or semi-maintained yet popular package (such as ansi-terminal) to pull off an attack.

The process does protect against *targeted* malicious takeover (i.e. an adversary wants to inject malicious code in exactly the (maintained) package K, and not just any moderately popular (but unmaintained) package L). But that is not enough. So security and trust have to be enforced by a different mechanism anyway.

Roman

* Evan Cofsky [2014-01-31 06:11:52+0000]

...

Hello,

As you all know I'm new to Haskell, but not at all new to software communities or computer security. The Haskell community is maturing, and as part of that we will have to be able to ensure that social engineering attacks against our core infrastructure, such as the canonical package repository, are not trivial to pull off successfully.

There are several instances just in the past week that have been making the rounds of basic impersonation attacks having disastrous consequences because people were quite willing to trust complete strangers, and of course we all know about the various problems faced by the Node and the Ruby communities and their package management systems, which were fully automated.

I put a lot more faith into Haskell programs, probably naively so, but as people, though, we have to at least maintain a certain amount of healthy skepticism when it comes to responding to requests to take over projects, and without a doubt along with that skepticism comes friction and process. But these are absolutely essential to having a trustworthy resource in Hackage, and I think that far outweighs the ficticious freedoms offered by management by handshake and a smile, especially for such an important community resource.

-- Evan Cofsky

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

On Fri, Jan 31, 2014 at 7:22 AM, Erik Hesselink wrote:

On Fri, Jan 31, 2014 at 1:12 PM, Roman Cheplyaka wrote:

...

...

Again, do you have any suggestions to make things better?

Here I merely want people to realize that there is a problem. How to solve it is a whole new discussion.

I think plenty of people (including me) have already agreed that there is a problem. So I don't understand the point of your message about security, then.

It was a response to Evan Coskey, who introduced a bit of a diversion. -- brandon s allbery kf8nh sine nomine associates allbery.b@gmail.com ballbery@sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net

On Fri, Jan 31, 2014 at 11:10 AM, Evan Cofsky wrote:

Well sorry to have introduced a diversion to you boys talking about

Someone who has managed to conflate confusion with disinterest, and has an attitude problem on top of it, perhaps needs to be spanked. -- brandon s allbery kf8nh sine nomine associates allbery.b@gmail.com ballbery@sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net

People are missing a key point: hackage packages are append only. Any upload will not override any prior version, and a bad new version is quite easy to deprecate. I'm not sure I'm comfortable with the idea of trustees having super upload powers by default (Speaking as the only person with trustee but not admin powers). Ie Id want a "trustee" upload to be a distinguished API thst I couldn't trip using cabal upload and if such a hypothetical power existed, I'd probably solicit feedback from a few folks by emailing the libraries list and testing any such upload locally. That aside: why isn't anyone helping work on hackage-server? We really need a few Heros to help work on hackage server. Otherwise it's kinda moot! :-) On Friday, January 31, 2014, Brandon Allbery wrote:

On Fri, Jan 31, 2014 at 7:22 AM, Erik Hesselink javascript:_e(%7B%7D,'cvml','hesselink@gmail.com');

...

wrote:

...

On Fri, Jan 31, 2014 at 1:12 PM, Roman Cheplyaka javascript:_e(%7B%7D,'cvml','roma@ro-che.info');> wrote:

...

...

Again, do you have any suggestions to make things better?

Here I merely want people to realize that there is a problem. How to solve it is a whole new discussion.

I think plenty of people (including me) have already agreed that there is a problem. So I don't understand the point of your message about security, then.

It was a response to Evan Coskey, who introduced a bit of a diversion.

-- brandon s allbery kf8nh sine nomine associates allbery.b@gmail.com javascript:_e(%7B%7D,'cvml','allbery.b@gmail.com'); ballbery@sinenomine.netjavascript:_e(%7B%7D,'cvml','ballbery@sinenomine.net'); unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net

* Erik Hesselink [2014-01-31 10:04:33+0100]

I propose to make the trustees group able to upload any package, with the understanding that they only do so to make packages where the maintainer is unreachable compile on more compilers or with more versions of dependencies. The newly uploaded version should have a public repository of the forked source available and listed in the cabal file. The process would then be:

* User fixes a package, emails the maintainer. * No response: User emails trustees. * Trustees check the above conditions, and upload the new version.

This is more lightweight that the process to take over maintainership, and it can be, because we're not trusting a random user with a random package. Instead, we're only trusting a fixed set of maintainers and a small, publicly visible change. Because of this, the waiting times for non-responsiveness can probably also be shorter than in the maintainer take-over process.

Would this alleviate the frustration, while at the same time maintaining enough security and sense of package ownership?

It could. Let's implement it and see how it goes. Roman

On Fri, 31 Jan 2014 10:04:33 +0100, Erik Hesselink writes:

* User fixes a package, emails the maintainer. * No response: User emails trustees. * Trustees check the above conditions, and upload the new version.

* Attacker "fixes the package", emails the maintainer with a typo in the email address (if the package is really unmaintained and the maintainer is unreachable this typo trick is not even necessary) * No response: attacker emails trustees * Attacker provides a github repository where the last commit is nice, but the attack is in previous commits that are converted from darcs to git(hub) Of course I'd never attack my beloved Haskell community, but I also don't believe in snake-oil processes. Gergely

On 01/31/2014 02:21 PM, Erik Hesselink wrote:

Yes, if there's no original repo to compare against, you can probably fake a lot of stuff. I cannot see how to easily guard against this,

Maybe have people provide a diff against the latest version instead of a repo/tarball ? That is easy to generate, review, and apply.

On Fri, Jan 31, 2014 at 5:21 PM, Erik Hesselink wrote:

On Fri, Jan 31, 2014 at 1:55 PM, Gergely Risko wrote:

...

On Fri, 31 Jan 2014 10:04:33 +0100, Erik Hesselink writes:

...

* User fixes a package, emails the maintainer. * No response: User emails trustees. * Trustees check the above conditions, and upload the new version.

* Attacker "fixes the package", emails the maintainer with a typo in the email address (if the package is really unmaintained and the maintainer is unreachable this typo trick is not even necessary) * No response: attacker emails trustees * Attacker provides a github repository where the last commit is nice, but the attack is in previous commits that are converted from darcs to git(hub)

Yes, if there's no original repo to compare against, you can probably fake a lot of stuff. I cannot see how to easily guard against this, without making the process more cumbersome.

Well, surely we can (and should!) compare the given "new" repository with the latest hackage version. Comparing against the canonical repository can lead to problems if the canonical repository contains commits that have not been released to Hackage but which introduce breaking changes, for example.

Perhaps it was wrong of me to mention security at all. But having the concept of maintainers (and thus *some* process for changing these) still makes a lot of sense to me with regard to 'ownership' of a package. Should we abolish that and go back to the situation of no ownership/maintainership checks? Or should we skip checking the source code?

Regards,

Erik _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

-- Sincerely yours, -- Daniil

We could actually partially automate this: 1) Package maintainership switch is submitted online, with a new replacement package, and perhaps a message. 2) An email is sent to the maintainer with a link to either: - delete the replacement package - allow one-time upload - permanently add the uploader as a maintainer - permanently switch maintaners to the uploader 3) While the package is in this limbo state waiting for a response from the maintainer, put a link to the package at the bottom of the hackage page in a new "suggested replacements" section. In this section, each candidate replacement package is listed, along with its message and how long it's been waiting. 4) After a bikeshed-long amount of time with no response from the maintainer (I'll suggest 1 month), the package is automatically updated to the suggested version and the package uploader is added as a maintainer. On Fri, Jan 31, 2014 at 11:34 AM, Daniil Frumin wrote:

I think the proposed approach is only reasonable. However, I would like to stress that in any case it would be better to make sure that we give the maintainer enough time to respond, e.g.: if the maintainer is unreachable for a couple of weeks at least

On Fri, Jan 31, 2014 at 1:04 PM, Erik Hesselink wrote:

...

On Fri, Jan 31, 2014 at 3:15 AM, Roman Cheplyaka wrote:

...

* Erik de Castro Lopo [2014-01-31 09:22:36+1100]

...

I really can understand why you did this; I am frustrated by some of the same issues. However, I think if any significant number of people did this, the results could easily be disasterous.

Agreed. Maybe we need those disasterous results to realize that the current process is bad and come up with a better one. Or maybe it's just me, and everyone else is happy (enough) with the process, so nothing will happen.

That's a rather fatalist attitude, and also one that is not warranted given the replies in this thread. Let me try to be more constructive instead:

I propose to make the trustees group able to upload any package, with the understanding that they only do so to make packages where the maintainer is unreachable compile on more compilers or with more versions of dependencies. The newly uploaded version should have a public repository of the forked source available and listed in the cabal file. The process would then be:

* User fixes a package, emails the maintainer. * No response: User emails trustees. * Trustees check the above conditions, and upload the new version.

This is more lightweight that the process to take over maintainership, and it can be, because we're not trusting a random user with a random package. Instead, we're only trusting a fixed set of maintainers and a small, publicly visible change. Because of this, the waiting times for non-responsiveness can probably also be shorter than in the maintainer take-over process.

Would this alleviate the frustration, while at the same time maintaining enough security and sense of package ownership?

Regards,

Erik _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

-- Sincerely yours, -- Daniil _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

-- Clark. Key ID : 0x78099922 Fingerprint: B292 493C 51AE F3AB D016 DD04 E5E3 C36F 5534 F907

Fair point. I wish I could, but the soonest I could start checking it out is in ~3 months. - Clark On Fri, Jan 31, 2014 at 11:44 AM, Carter Schonwald < carter.schonwald@gmail.com> wrote:

Problem: no one is really actively working on hackage-server. Are you volunteering? :-)

On Friday, January 31, 2014, Clark Gaebel wrote:

...

We could actually partially automate this:

1) Package maintainership switch is submitted online, with a new replacement package, and perhaps a message. 2) An email is sent to the maintainer with a link to either: - delete the replacement package - allow one-time upload - permanently add the uploader as a maintainer - permanently switch maintaners to the uploader 3) While the package is in this limbo state waiting for a response from the maintainer, put a link to the package at the bottom of the hackage page in a new "suggested replacements" section. In this section, each candidate replacement package is listed, along with its message and how long it's been waiting. 4) After a bikeshed-long amount of time with no response from the maintainer (I'll suggest 1 month), the package is automatically updated to the suggested version and the package uploader is added as a maintainer.

On Fri, Jan 31, 2014 at 11:34 AM, Daniil Frumin wrote:

...

I think the proposed approach is only reasonable. However, I would like to stress that in any case it would be better to make sure that we give the maintainer enough time to respond, e.g.: if the maintainer is unreachable for a couple of weeks at least

On Fri, Jan 31, 2014 at 1:04 PM, Erik Hesselink wrote:

...

On Fri, Jan 31, 2014 at 3:15 AM, Roman Cheplyaka wrote:

...

* Erik de Castro Lopo [2014-01-31 09:22:36+1100]

...

I really can understand why you did this; I am frustrated by some of the same issues. However, I think if any significant number of people did this, the results could easily be disasterous.

Agreed. Maybe we need those disasterous results to realize that the current process is bad and come up with a better one. Or maybe it's just me, and everyone else is happy (enough) with the process, so nothing will happen.

That's a rather fatalist attitude, and also one that is not warranted given the replies in this thread. Let me try to be more constructive instead:

I propose to make the trustees group able to upload any package, with the understanding that they only do so to make packages where the maintainer is unreachable compile on more compilers or with more versions of dependencies. The newly uploaded version should have a public repository of the forked source available and listed in the cabal file. The process would then be:

* User fixes a package, emails the maintainer. * No response: User emails trustees. * Trustees check the above conditions, and upload the new version.

This is more lightweight that the process to take over maintainership, and it can be, because we're not trusting a random user with a random package. Instead, we're only trusting a fixed set of maintainers and a small, publicly visible change. Because of this, the waiting times for non-responsiveness can probably also be shorter than in the maintainer take-over process.

Would this alleviate the frustration, while at the same time maintaining enough security and sense of package ownership?

Regards,

Erik _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

-- Sincerely yours, -- Daniil _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

-- Clark.

Key ID : 0x78099922 Fingerprint: B292 493C 51AE F3AB D016 DD04 E5E3 C36F 5534 F907

-- Clark. Key ID : 0x78099922 Fingerprint: B292 493C 51AE F3AB D016 DD04 E5E3 C36F 5534 F907

I have a problem with the 4th step. What if maintainer is unreachable, but the updated version of the package is broken/breaking ever dependency? What if there are several replacements awaiting? I personally think that problem we are facing is not technical, but a social one. Call me old fashioned, but I prefer trustees to the automatic mechanism. I understand that Roman may have been really irritated by the whole process - but on the other hand, do we really need/want the process of overtaking packages to be easy? I strongly align with Gershom's position. We should make the process more transparent and visible. In order to put my money where my mouth is, I created a wiki page that (hopefully) describes the process of taking over a package: http://www.haskell.org/haskellwiki/Taking_over_a_package You are strongly encouraged to edit that page and give more details (especially given my far from perfect English) Maybe it is a good idea to have links to that wiki article on every package page on Hackage? On Fri, Jan 31, 2014 at 8:44 PM, Carter Schonwald wrote:

Problem: no one is really actively working on hackage-server. Are you volunteering? :-)

On Friday, January 31, 2014, Clark Gaebel wrote:

...

We could actually partially automate this:

1) Package maintainership switch is submitted online, with a new replacement package, and perhaps a message. 2) An email is sent to the maintainer with a link to either: - delete the replacement package - allow one-time upload - permanently add the uploader as a maintainer - permanently switch maintaners to the uploader 3) While the package is in this limbo state waiting for a response from the maintainer, put a link to the package at the bottom of the hackage page in a new "suggested replacements" section. In this section, each candidate replacement package is listed, along with its message and how long it's been waiting. 4) After a bikeshed-long amount of time with no response from the maintainer (I'll suggest 1 month), the package is automatically updated to the suggested version and the package uploader is added as a maintainer.

On Fri, Jan 31, 2014 at 11:34 AM, Daniil Frumin wrote:

...

I think the proposed approach is only reasonable. However, I would like to stress that in any case it would be better to make sure that we give the maintainer enough time to respond, e.g.: if the maintainer is unreachable for a couple of weeks at least

On Fri, Jan 31, 2014 at 1:04 PM, Erik Hesselink wrote:

...

On Fri, Jan 31, 2014 at 3:15 AM, Roman Cheplyaka wrote:

...

* Erik de Castro Lopo [2014-01-31 09:22:36+1100]

...

I really can understand why you did this; I am frustrated by some of the same issues. However, I think if any significant number of people did this, the results could easily be disasterous.

Agreed. Maybe we need those disasterous results to realize that the current process is bad and come up with a better one. Or maybe it's just me, and everyone else is happy (enough) with the process, so nothing will happen.

That's a rather fatalist attitude, and also one that is not warranted given the replies in this thread. Let me try to be more constructive instead:

I propose to make the trustees group able to upload any package, with the understanding that they only do so to make packages where the maintainer is unreachable compile on more compilers or with more versions of dependencies. The newly uploaded version should have a public repository of the forked source available and listed in the cabal file. The process would then be:

* User fixes a package, emails the maintainer. * No response: User emails trustees. * Trustees check the above conditions, and upload the new version.

This is more lightweight that the process to take over maintainership, and it can be, because we're not trusting a random user with a random package. Instead, we're only trusting a fixed set of maintainers and a small, publicly visible change. Because of this, the waiting times for non-responsiveness can probably also be shorter than in the maintainer take-over process.

Would this alleviate the frustration, while at the same time maintaining enough security and sense of package ownership?

Regards,

Erik _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

-- Sincerely yours, -- Daniil _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

-- Clark.

Key ID : 0x78099922 Fingerprint: B292 493C 51AE F3AB D016 DD04 E5E3 C36F 5534 F907

-- Sincerely yours, -- Daniil

There could be an email made to the relevant mailing lists during a takeover attempt. That way we get human visibility, human "veto power" if the email goes to libraries@, and automation when there are no objections. On Fri, Jan 31, 2014 at 12:09 PM, Carter Schonwald < carter.schonwald@gmail.com> wrote:

Agreed. It should not be automatic. There should be lots of human visible interaction publicly going on.

On Friday, January 31, 2014, Daniil Frumin wrote:

...

I have a problem with the 4th step. What if maintainer is unreachable, but the updated version of the package is broken/breaking ever dependency? What if there are several replacements awaiting?

I personally think that problem we are facing is not technical, but a social one. Call me old fashioned, but I prefer trustees to the automatic mechanism.

I understand that Roman may have been really irritated by the whole process - but on the other hand, do we really need/want the process of overtaking packages to be easy? I strongly align with Gershom's position. We should make the process more transparent and visible. In order to put my money where my mouth is, I created a wiki page that (hopefully) describes the process of taking over a package: http://www.haskell.org/haskellwiki/Taking_over_a_package You are strongly encouraged to edit that page and give more details (especially given my far from perfect English)

Maybe it is a good idea to have links to that wiki article on every package page on Hackage?

...

Problem: no one is really actively working on hackage-server. Are you volunteering? :-)

On Friday, January 31, 2014, Clark Gaebel wrote:

...

We could actually partially automate this:

1) Package maintainership switch is submitted online, with a new replacement package, and perhaps a message. 2) An email is sent to the maintainer with a link to either: - delete the replacement package - allow one-time upload - permanently add the uploader as a maintainer - permanently switch maintaners to the uploader 3) While the package is in this limbo state waiting for a response from the maintainer, put a link to the package at the bottom of the hackage

...

...

in a new "suggested replacements" section. In this section, each candidate replacement package is listed, along with its message and how long it's been waiting. 4) After a bikeshed-long amount of time with no response from the maintainer (I'll suggest 1 month), the package is automatically updated to the suggested version and the package uploader is added as a

...

...

On Fri, Jan 31, 2014 at 11:34 AM, Daniil Frumin wrote:

...

I think the proposed approach is only reasonable. However, I would like to stress that in any case it would be better to make sure that we give the maintainer enough time to respond, e.g.: if the maintainer is unreachable for a couple of weeks at least

On Fri, Jan 31, 2014 at 1:04 PM, Erik Hesselink wrote:

...

On Fri, Jan 31, 2014 at 3:15 AM, Roman Cheplyaka wrote: > * Erik de Castro Lopo [2014-01-31 > 09:22:36+1100] >> I really can understand why you did this; I am frustrated by some

of

...

...

...

>> the same issues. However, I think if any significant number of

...

...

...

...

>> did this, the results could easily be disasterous. > > Agreed. Maybe we need those disasterous results to realize that the > current process is bad and come up with a better one. Or maybe it's > just > me, and everyone else is happy (enough) with the process, so nothing > will happen.

That's a rather fatalist attitude, and also one that is not warranted given the replies in this thread. Let me try to be more constructive instead:

I propose to make the trustees group able to upload any package, with the understanding that they only do so to make packages where the maintainer is unreachable compile on more compilers or with more versions of dependencies. The newly uploaded version should have a public repository of the forked source available and listed in the cabal file. The process would then be:

* User fixes a package, emails the maintainer. * No response: User emails trustees. * Trustees check the above conditions, and upload the new version.

This is more lightweight that the process to take over

On Fri, Jan 31, 2014 at 8:44 PM, Carter Schonwald wrote: page maintainer. people maintainership,

...

...

...

...

and it can be, because we're not trusting a random user with a random package. Instead, we're only trusting a fixed set of maintainers and a small, publicly visible change. Because of this, the waiting times for non-respo

-- Clark. Key ID : 0x78099922 Fingerprint: B292 493C 51AE F3AB D016 DD04 E5E3 C36F 5534 F907

The automation could just be "new package version + new maintainer added in 30 days if no one manually stops it". Takeovers should be easy to stop for both existing library maintainers and any core "trusted" members of the community. On Fri, Jan 31, 2014 at 1:03 PM, Carter Schonwald < carter.schonwald@gmail.com> wrote:

How would the automation work? Automation in trust models is very very fiddly to get right. Like really hard. Like a research problem with reality consequences hard.

At the end of the day, it's a people problem. The best we can do is come up with a very audit able, publicly visible process that makes everything easy for 3rd partiies to audit. And prevents / catches any abuse before it does something like break vector or ByteString.

On Friday, January 31, 2014, Clark Gaebel wrote:

...

There could be an email made to the relevant mailing lists during a takeover attempt. That way we get human visibility, human "veto power" if the email goes to libraries@, and automation when there are no objections.

On Fri, Jan 31, 2014 at 12:09 PM, Carter Schonwald < carter.schonwald@gmail.com> wrote:

Agreed. It should not be automatic. There should be lots of human visible interaction publicly going on.

On Friday, January 31, 2014, Daniil Frumin wrote:

I have a problem with the 4th step. What if maintainer is unreachable, but the updated version of the package is broken/breaking ever dependency? What if there are several replacements awaiting?

I personally think that problem we are facing is not technical, but a social one. Call me old fashioned, but I prefer trustees to the automatic mechanism.

I understand that Roman may have been really irritated by the whole process - but on the other hand, do we really need/want the process of overtaking packages to be easy? I strongly align with Gershom's position. We should make the process more transparent and visible. In order to put my money where my mouth is, I created a wiki page that (hopefully) describes the process of taking over a package: http://www.haskell.org/haskellwiki/Taking_over_a_package You are strongly encouraged to edit that page and give more details (especially given my far from perfect English)

Maybe it is a good idea to have links to that wiki article on every package page on Hackage?

...

Problem: no one is really actively working on hackage-server. Are you volunteering? :-)

On Friday, January 31, 2014, Clark Gaebel wrote:

...

We could actually partially automate this:

1) Package maintainership switch is submitted online, with a new replacement package, and perhaps a message. 2) An email is sent to the maintainer with a link to either: - delete the replacement package - allow one-time upload - permanently add the uploader as a maintainer - permanently switch maintaners to the uploader 3) While the package is in this limbo state waiting for a response from the maintainer, put a link to the package at the bottom of the hackage

...

...

in a new "suggested replacements" section. In this section, each candidate replacement package is listed, along with its message and how long it's been waiting. 4) After a bikeshed-long amount of time with no response from the maintainer (I'll suggest 1 month), the package is automatically updated to the suggested version and the package uploader is added as a

...

...

On Fri, Jan 31, 2014 at 11:34 AM, Daniil Frumin wrote:

...

I think the proposed approach is only reasonable. However, I would like to stress that in any case it would be better to make sure that we give the maintainer enough time to respond, e.g.: if the maintainer is unreachable for a couple of weeks at least

On Fri, Jan 31, 2014 at 1:04 PM, Erik Hesselink wrote:

...

On Fri, Jan 31, 2014 at 3:15 AM, Roman Cheplyaka wrote: > * Erik de Castro Lopo [2014-01-31 > 09:22:36+1100] >> I really can understand why you did this; I am frustrated by some

of

...

...

...

>> the same issues. However, I think if any significant number of

On Fri, Jan 31, 2014 at 8:44 PM, Carter Schonwald wrote: page maintainer. people

...

...

...

...

>> did this, the results could easily be disasterous. > > Agreed. Maybe we need those disasterous results to realize that the > current process is bad and come up with a better one. Or maybe it's > just > me, and everyone else is happy (enough) with the process, so nothing > will happen.

That's a rather fatalist attitude, and als

-- Clark.

Key ID : 0x78099922 Fingerprint: B292 493C 51AE F3AB D016 DD04 E5E3 C36F 5534 F907

-- Clark. Key ID : 0x78099922 Fingerprint: B292 493C 51AE F3AB D016 DD04 E5E3 C36F 5534 F907

Yes. It should be perhaps even require two trustees / admins / whatever if we are actually serious about making things proof against social engineering. Or something. Nb: hackage admins have the super powers. I'm just a trustee. Which just means I can delete bad doc builds to force em to rebuild. On Friday, January 31, 2014, Daniil Frumin wrote:

On Fri, Jan 31, 2014 at 10:05 PM, Clark Gaebel javascript:;> wrote:

...

The automation could just be "new package version + new maintainer added in 30 days if no one manually stops it".

Takeovers should be easy to stop for both existing library maintainers and any core "trusted" members of the community.

Rights, that's why we have hackage trustees. They can easily overwrite/update any package (if I understand the process correctly). Complete takeover of a package should *not* be easy, we must give the maintainers a good sense of ownership.

...

On Fri, Jan 31, 2014 at 1:03 PM, Carter Schonwald wrote:

...

How would the automation work? Automation in trust models is very very fiddly to get right. Like really hard. Like a research problem with reality consequences hard.

At the end of the day, it's a people problem. The best we can do is

come

...

...

up with a very audit able, publicly visible process that makes everything easy for 3rd partiies to audit. And prevents / catches any abuse before it does something like break vector or ByteString.

On Friday, January 31, 2014, Clark Gaebel wrote:

...

There could be an email made to the relevant mailing lists during a takeover attempt. That way we get human visibility, human "veto power"

if

...

the email goes to libraries@, and automation when there are no objections.

On Fri, Jan 31, 2014 at 12:09 PM, Carter Schonwald wrote:

Agreed. It should not be automatic. There should be lots of human visible interaction publicly going on.

On Friday, January 31, 2014, Daniil Frumin wrote:

I have a problem with the 4th step. What if maintainer is unreachable, but the updated version of the package is broken/breaking ever dependency? What if there are several replacements awaiting?

I personally think that problem we are facing is not technical, but a social one. Call me old fashioned, but I prefer trustees to the automatic mechanism.

I understand that Roman may have been really irritated by the whole process - but on the other hand, do we really need/want the process of overtaking packages to be easy? I strongly align with Gershom's position. We should make the process more transparent and visible. In order to put my money where my mouth is, I created a wiki page that (hopefully) describes the process of taking over a package: http://www.haskell.org/haskellwiki/Taking_over_a_package You are strongly encouraged to edit that page and give more details (especially given my far from perfect English)

Maybe it is a good idea to have links to that wiki article on every package page on Hackage?

On Fri, Jan 31, 2014 at 8:44 PM, Carter Schonwald wrote:

...

Problem: no one is really actively working on hackage-server. Are you volunteering? :-)

On Friday, January 31, 2014, Clark Gaebel wrote:

...

We could actually partially automate this:

1) Package maintainership switch is submitted online, with a new replacement package, and perhaps a message. 2) An email is sent to the maintainer with a link to either: - delete the replacement package - allow one-time upload - permanently add the uploader as a maintainer - permanently switch maintaners to the uploader 3) While the package is in this limbo state waiting for a response from the maintainer, put a link to the package at the bottom of the

hackage

...

pag-- Sincerely yours, -- Daniil

Fair enough. I'm also not sure the Haskell community has a resident security expert, so it would likely be bad security at best. - Clark On Fri, Jan 31, 2014 at 2:04 PM, Carter Schonwald < carter.schonwald@gmail.com> wrote:

automation ... isn't flexible. the moment theres an automatic path, a lot of non social engineering approaches to break any hackage trust model

On Fri, Jan 31, 2014 at 12:45 PM, Clark Gaebel wrote:

...

There could be an email made to the relevant mailing lists during a takeover attempt. That way we get human visibility, human "veto power" if the email goes to libraries@, and automation when there are no objections.

On Fri, Jan 31, 2014 at 12:09 PM, Carter Schonwald < carter.schonwald@gmail.com> wrote:

...

Agreed. It should not be automatic. There should be lots of human visible interaction publicly going on.

On Friday, January 31, 2014, Daniil Frumin wrote:

...

I have a problem with the 4th step. What if maintainer is unreachable, but the updated version of the package is broken/breaking ever dependency? What if there are several replacements awaiting?

I personally think that problem we are facing is not technical, but a social one. Call me old fashioned, but I prefer trustees to the automatic mechanism.

I understand that Roman may have been really irritated by the whole process - but on the other hand, do we really need/want the process of overtaking packages to be easy? I strongly align with Gershom's position. We should make the process more transparent and visible. In order to put my money where my mouth is, I created a wiki page that (hopefully) describes the process of taking over a package: http://www.haskell.org/haskellwiki/Taking_over_a_package You are strongly encouraged to edit that page and give more details (especially given my far from perfect English)

Maybe it is a good idea to have links to that wiki article on every package page on Hackage?

...

Problem: no one is really actively working on hackage-server. Are you volunteering? :-)

On Friday, January 31, 2014, Clark Gaebel wrote:

...

We could actually partially automate this:

1) Package maintainership switch is submitted online, with a new replacement package, and perhaps a message. 2) An email is sent to the maintainer with a link to either: - delete the replacement package - allow one-time upload - permanently add the uploader as a maintainer - permanently switch maintaners to the uploader 3) While the package is in this limbo state waiting for a response

from

...

the maintainer, put a link to the package at the bottom of the hackage page in a new "suggested replacements" section. In this section, each candidate replacement package is listed, along with its message and how long it's been waiting. 4) After a bikeshed-long amount of time with no response from the maintainer (I'll suggest 1 month), the package is automatically updated to the suggested version and the package uploader is added as a

...

...

On Fri, Jan 31, 2014 at 11:34 AM, Daniil Frumin wrote: > > I think the proposed approach is only reasonable. However, I would > like to stress that in any case it would be better to make sure that > we give the maintainer enough time to respond, e.g.: if the

...

...

> is unreachable for a couple of weeks at least > > On Fri, Jan 31, 2014 at 1:04 PM, Erik Hesselink < hesselink@gmail.com> > wrote: > > On Fri, Jan 31, 2014 at 3:15 AM, Roman Cheplyaka < roma@ro-che.info> > > wrote: > >> * Erik de Castro Lopo [2014-01-31 > >> 09:22:36+1100] > >>> I really can understand why you did this; I am frustrated by some of > >>> the same issues. However, I think if any significant number of

...

...

> >>> did this, the results could easily be disasterous. > >> > >> Agreed. Maybe we need those disasterous results to realize that

...

...

> >> current process is bad and come up with a better one. Or maybe it's > >> just > >> me, and everyone else is happy (enough) with the process, so nothing > >> will happen. > > > > That's a rather fatalist attitude, and also one that is not warranted > > given the replies in this thread. Let me try to be more constructive > > instead: > > > > I propose to make the trustees group able to upload any package, with > > the understanding that they only do so to make packages where the > > maintainer is unreachable compile on more compilers or with more > > versions of dependencies. The newly uploaded version should have a > > public repository of the forked source available and listed in the > > cabal file. The process would then be: > > > > * User fixes a package, emails the maintainer. > > * No response: User emails trustees. > > * Trustees check the above conditions, and upload the new version. > > > > This is more lightweight that the process to take over

On Fri, Jan 31, 2014 at 8:44 PM, Carter Schonwald wrote: maintainer. maintainer people the maintainership,

...

...

> > and it can be, because we're not trusting a random user with a random > > package. Instead, we're only trusting a fixed set of maintainers and a > > small, publicly visible change. Because of this, the waiting times for > > non-respo

-- Clark.

Key ID : 0x78099922 Fingerprint: B292 493C 51AE F3AB D016 DD04 E5E3 C36F 5534 F907

-- Clark. Key ID : 0x78099922 Fingerprint: B292 493C 51AE F3AB D016 DD04 E5E3 C36F 5534 F907

* Sergei Trofimovich [2014-02-09 10:00:21+0300]

Now as regex-tdfa-1.2.0 is on hackage.

What status of your regex-tdfa-rc?

Thanks for the reminder. I just marked it as deprecated in favor of regex-tdfa.

May I bug you with performance problems compared to regex-dfa?

No, I don't know much about how it works. My sole reason for forking was to keep it buildable. Roman