Re: [Haskell-cafe] Improvements to package hosting and security

Define "superior"?
As argued in the proposal, the salient features are that by devolving
practically everything to Git+GPG, we end up with less code to maintain in
our tooling, less code to maintain in our infrastructure (namely
hackage-server), a more reliable service, and a smaller chance of buggering
up security related activities (such as signing and managing trust).
We're not introducing dependencies on dynamically linked system libraries
that makes tooling hard to distribute. We're not asking users to install
anything new that isn't already a staple of most developer desktops, and
not asking users, Hackage trustees and Hackage admins to manage new
identities with new key formats that aren't the existing ones they already
have (namely GnuPG). Further, users can still opt-out of signature
verification if they want to.
Compared to alternative approaches - there has been a proposal to get
incremental updates à la Git differently by growing (potentially
infinitely) the end of a tar file served by the server via HTTP. This means
grabbing the history for new package revisions cannot be opted out from
easily. With Git, you get this for free, since users can `git clone
--depth=1` and still be able to do a `git pull` later and verify
signatures. You further get the advantage of being able to directly mine
the history of changes, using standard tools, something that can't be done
directly on the tar file without more custom tooling (or post conversion to
Git).
On 28 April 2015 at 23:33, Bardur Arantsson
On 28-04-2015 23:09, Mathieu Boespflug wrote:
[removing erroneous haskell-cafe@googlegroups.com from To list.]
(I'm not the person you're responing to. From the mail-headers, I can't see the person(s) you're responding to, but so be it.)
Do you have evidence that your approach is superior, and could you please cite it? [Or, alternatively provide negative evidence for $OTHER_APPROACH.])
Regards,
-- You received this message because you are subscribed to the Google Groups "Commercial Haskell" group. To unsubscribe from this group and stop receiving emails from it, send an email to commercialhaskell+unsubscribe@googlegroups.com. To post to this group, send email to commercialhaskell@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/commercialhaskell/mhouaf%24it2%241%40ger.g... . For more options, visit https://groups.google.com/d/optout.

Mathieu Boespflug-4 wrote
We're not introducing dependencies on dynamically linked system libraries that makes tooling hard to distribute. We're not asking users to install anything new that isn't already a staple of most developer desktops
My sole concern with this is that git is often not present on build servers, which may be minimal cloud VMs. Here's what I get when I try to install git on mine: # apt install git --no-install-recommends ... The following NEW packages will be installed: git git-man libcurl3-gnutls liberror-perl libexpat1 libgdbm3 perl perl-modules 0 upgraded, 8 newly installed, 0 to remove and 2 not upgraded. Need to get 10.4 MB of archives. After this operation, 57.2 MB of additional disk space will be used. Not unbearable, but not insignificant either. -- View this message in context: http://haskell.1045720.n5.nabble.com/Improvements-to-package-hosting-and-sec... Sent from the Haskell - Haskell-Cafe mailing list archive at Nabble.com.

On Thu, Apr 30, 2015 at 10:08 AM Jeremy
Mathieu Boespflug-4 wrote
We're not introducing dependencies on dynamically linked system libraries that makes tooling hard to distribute. We're not asking users to install anything new that isn't already a staple of most developer desktops
My sole concern with this is that git is often not present on build servers, which may be minimal cloud VMs. Here's what I get when I try to install git on mine:
# apt install git --no-install-recommends ... The following NEW packages will be installed: git git-man libcurl3-gnutls liberror-perl libexpat1 libgdbm3 perl perl-modules 0 upgraded, 8 newly installed, 0 to remove and 2 not upgraded. Need to get 10.4 MB of archives. After this operation, 57.2 MB of additional disk space will be used.
Not unbearable, but not insignificant either.
One possible workflow[1] would be to have a dedicated system that uses Git and GPG to pull the current versions of all packages and verify signatures. That system could then create a snapshot of that information that could simply be downloaded by a build server. In fact, there could even be a public server available providing that functionality, with the caveat that- like today- you'd need to trust that server to not be compromised. I think this is what Mathieu was getting at when he said:
Further, users can still opt-out of signature verification if they want to.
Michael [1] And possible may be too weak a word, as I have an implementation pretty close to this already.

On 30-04-2015 09:21, Michael Snoyman wrote:
On Thu, Apr 30, 2015 at 10:08 AM Jeremy
wrote: Mathieu Boespflug-4 wrote
We're not introducing dependencies on dynamically linked system libraries that makes tooling hard to distribute. We're not asking users to install anything new that isn't already a staple of most developer desktops
My sole concern with this is that git is often not present on build servers, which may be minimal cloud VMs. Here's what I get when I try to install git on mine:
# apt install git --no-install-recommends ... The following NEW packages will be installed: git git-man libcurl3-gnutls liberror-perl libexpat1 libgdbm3 perl perl-modules 0 upgraded, 8 newly installed, 0 to remove and 2 not upgraded. Need to get 10.4 MB of archives. After this operation, 57.2 MB of additional disk space will be used.
Not unbearable, but not insignificant either.
One possible workflow[1] would be to have a dedicated system that uses Git and GPG to pull the current versions of all packages and verify signatures. That system could then create a snapshot of that information that could simply be downloaded by a build server. In fact, there could even be a public server available providing that functionality, with the caveat that- like today- you'd need to trust that server to not be compromised.
Isn't this just another moving part? (Moving parts are generally considered bad news when you're trying to engineer a secure system.) And has there been any review of git wrt. if it is robust to malicious servers? (E.g. if I do a "git fetch", can the server I happen to be talking to just spew data at the client indefinitely to, for example, fill up its disk or to prevent it from ever progressing?) Regards,

On 30/04/2015 08:08, Jeremy wrote:
Mathieu Boespflug-4 wrote
We're not introducing dependencies on dynamically linked system libraries that makes tooling hard to distribute. We're not asking users to install anything new that isn't already a staple of most developer desktops My sole concern with this is that git is often not present on build servers, which may be minimal cloud VMs. Here's what I get when I try to install git on mine:
# apt install git --no-install-recommends ... The following NEW packages will be installed: git git-man libcurl3-gnutls liberror-perl libexpat1 libgdbm3 perl perl-modules 0 upgraded, 8 newly installed, 0 to remove and 2 not upgraded. Need to get 10.4 MB of archives. After this operation, 57.2 MB of additional disk space will be used.
I'm no fan of unnecessary bloat, but considering that typical Debian installation of ghc is +500MB (7.4) to +800M (7.10) (severely bloated by multiple almost copies of the same file), I'm not sure how relevant this is on a haskell build server. Plus, many people already uses git on build servers making a zero cost for most of us. -- Vincent

Vincent Hanquez wrote
On 30/04/2015 08:08, Jeremy wrote:
My sole concern with this is that git is often not present on build servers, which may be minimal cloud VMs. Here's what I get when I try to install git on mine:
# apt install git --no-install-recommends ... The following NEW packages will be installed: git git-man libcurl3-gnutls liberror-perl libexpat1 libgdbm3 perl perl-modules 0 upgraded, 8 newly installed, 0 to remove and 2 not upgraded. Need to get 10.4 MB of archives. After this operation, 57.2 MB of additional disk space will be used.
I'm no fan of unnecessary bloat, but considering that typical Debian installation of ghc is +500MB (7.4) to +800M (7.10) (severely bloated by multiple almost copies of the same file), I'm not sure how relevant this is on a haskell build server.
My entire server is under 150Mb (docker image). I build GHC myself without the bloat. -- View this message in context: http://haskell.1045720.n5.nabble.com/Improvements-to-package-hosting-and-sec... Sent from the Haskell - Haskell-Cafe mailing list archive at Nabble.com.

Ignore that, cosmic rays flipped some bits in my memory, server footprint is 418Mb (including cabal-install). -- View this message in context: http://haskell.1045720.n5.nabble.com/Improvements-to-package-hosting-and-sec... Sent from the Haskell - Haskell-Cafe mailing list archive at Nabble.com.
participants (5)
-
Bardur Arantsson
-
Jeremy
-
Mathieu Boespflug
-
Michael Snoyman
-
Vincent Hanquez