RE: hackage, cabal-get, and security

newer
wash, ghc 6.4

older
hackage, cabal-get, and security

Simon Marlow

17 May 2005 17 May '05

2:40 p.m.

On 17 May 2005 15:19, S. Alexander Jacobson wrote:

...

My solution in SearchPath (released yesterday) is to rely on a combination of HTTPS and links.

I don't want to start another huge debate, but I think your solution at least lacks support for some fundamental (IMO) features: - versioning - pre-compiled libraries - FFI (external C libraries & bundled C code) What's your story for these? e.g. what you do intend to happen when someone imports Graphics.UI.WX? Cheers, Simon

...

The user selects module directories that they trust. They access that directory via HTTPS. The module directory maps module names to package/module URLs. The user can decide that they only want to retrieve via HTTPS.

This is effectively a web of trust model, but the only key management needed is getting an SSL cert from a CA like Verisign or Thawte.

The open question here is whether it is easier to convince people to serve modules via HTTPS web servers or whether they prefer gnupg key management. A reason to believe that the former will be preferable to the later is that people can easily delegate SSL hosting to others. Delegating gnupg key management is non-trivial.

-Alex-

______________________________________________________________ S. Alexander Jacobson tel:917-770-6565 http://alexjacobson.com

On Mon, 16 May 2005, Isaac Jones wrote:

...
At the request of Dominic Steinitz, I'll outline the threats that I think this proposal protects against.

The signing of packages prevents a number of attacks between the packager and the server:

1) Accidentally or purposely hijacking a package that is signed by (belongs to) someone else.

2) Uploading a malicious package to replace someone else's good package.

3) Man-in-the-middle attcks between the packager and Hackage.

Checking signatures on the client side prevents:

1) Man-in-the-middle attcks between hackage and the client

2) Automatic installation of anonymous malicious packages

Building a trusted network of keys prevents:

1) Someone creating a key pretending to be someone else

2) Unchecked anonymous uploads (running arbitrary code from an unknown source)

One question that comes up is: how does the so-called "web of trust" help out with this situation? The signing of keys ties the identity of an individual (via their state-issued identification) to a particular key. Now if someone attempts one of the above attacks, after being "trusted" we know who they are in real life. So it's not really a "web of trust" but more like a "web of identity". We will need to put procedures in place for handling a variety of situations, like loss of trust, etc.

This proposal doesn't cover all of that, but it puts a bit of structure into place to raise the bar for an attacker sufficiently high in my opinion, and gives the end-users the tools they need to be as paranoid as they care to be.

peace,

isaac _______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

Show replies by date

S. Alexander Jacobson

17 May 17 May

4:14 p.m.

New subject: SearchPath (was RE: hackage, cabal-get, and security)

...

- versioning

With SearchPath, you supply the URLs of the module maps you want to use. SearchPath then uses these map to locate, download and "install" needed libraries without further user intervention. A module map is just a file mapping module names to the base URLs of directories in which they reside. If you want to supply the URL of a particular version of a module map rather than "latest" you are free to do that. Since module maps just point to the URLs of basedirs, you can have them point either to darcs/svn repositories of projects that serve out the latest version or to static directories holding a particular version. SearchPath doesn't yet support explicit WebDAV or darcs version identifiers, but I hope to get it added soon. Then the step of maintaining static checked out versions will become unnecessary.

...

- pre-compiled libraries - FFI (external C libraries & bundled C code)

Right now SearchPath only supports module directory base URLs in map files. I'd love for it to support automatic download and build for Cabal file and package URLs as well. I just don't know how to do it. Perhaps you guys can help on this? -Alex- ______________________________________________________________ S. Alexander Jacobson tel:917-770-6565 http://alexjacobson.com On Tue, 17 May 2005, Simon Marlow wrote:

...

On 17 May 2005 15:19, S. Alexander Jacobson wrote:

...
My solution in SearchPath (released yesterday) is to rely on a combination of HTTPS and links.

I don't want to start another huge debate, but I think your solution at least lacks support for some fundamental (IMO) features:

- versioning - pre-compiled libraries - FFI (external C libraries & bundled C code)

What's your story for these? e.g. what you do intend to happen when someone imports Graphics.UI.WX?

Cheers, Simon

...
The user selects module directories that they trust. They access that directory via HTTPS. The module directory maps module names to package/module URLs. The user can decide that they only want to retrieve via HTTPS.

This is effectively a web of trust model, but the only key management needed is getting an SSL cert from a CA like Verisign or Thawte.

The open question here is whether it is easier to convince people to serve modules via HTTPS web servers or whether they prefer gnupg key management. A reason to believe that the former will be preferable to the later is that people can easily delegate SSL hosting to others. Delegating gnupg key management is non-trivial.

-Alex-

______________________________________________________________ S. Alexander Jacobson tel:917-770-6565 http://alexjacobson.com

On Mon, 16 May 2005, Isaac Jones wrote:

...
At the request of Dominic Steinitz, I'll outline the threats that I think this proposal protects against.

The signing of packages prevents a number of attacks between the packager and the server:

1) Accidentally or purposely hijacking a package that is signed by (belongs to) someone else.

2) Uploading a malicious package to replace someone else's good package.

3) Man-in-the-middle attcks between the packager and Hackage.

Checking signatures on the client side prevents:

1) Man-in-the-middle attcks between hackage and the client

2) Automatic installation of anonymous malicious packages

Building a trusted network of keys prevents:

1) Someone creating a key pretending to be someone else

2) Unchecked anonymous uploads (running arbitrary code from an unknown source)

One question that comes up is: how does the so-called "web of trust" help out with this situation? The signing of keys ties the identity of an individual (via their state-issued identification) to a particular key. Now if someone attempts one of the above attacks, after being "trusted" we know who they are in real life. So it's not really a "web of trust" but more like a "web of identity". We will need to put procedures in place for handling a variety of situations, like loss of trust, etc.

This proposal doesn't cover all of that, but it puts a bit of structure into place to raise the bar for an attacker sufficiently high in my opinion, and gives the end-users the tools they need to be as paranoid as they care to be.

peace,

isaac _______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

** CRM114 Whitelisted by: ijones@syntaxpolice.org **

Isaac Jones

4:44 p.m.

New subject: SearchPath

"S. Alexander Jacobson" writes: (snip)

...

Right now SearchPath only supports module directory base URLs in map files. I'd love for it to support automatic download and build for Cabal file and package URLs as well. I just don't know how to do it. Perhaps you guys can help on this?

You'd just need a way to map module names to package names / versions. One thing you could do is add that kind of lookup to hackage's XML-RPC interface, shouldn't be too hard. Then you can query hackage "what package does this module belong to?"; it'll return a list of package identifiers (meaning package names / versions) and you' could pick among them, download, unpack, and install. I'd be fine with adding this to the xml-rpc interface if you want. Hopefully Lemmih will speak up if he thinks that's a really bad idea. Hackage can be gotten here: darcs get http://www.scannedinavian.org/~lemmih/hackage/ with dependencies here: http://www.scannedinavian.org/~lemmih/packages/ peace, isaac

S. Alexander Jacobson

6:32 p.m.

New subject: SearchPath

On Tue, 17 May 2005, Isaac Jones wrote:

...

You'd just need a way to map module names to package names / versions.

Actually, that's the easy part. For example, right now my test map file is: HAppS http://happs.org *.*.HaXml http://www.cs.york.ac.uk/fp/darcs/HaXml/src So, any module that starts with HAppS can be found at HAppS.org. And any module with HaXml at the third level of hierarchy can be found in the York HaXml darcs repository. e.g. it does this sort of transformation: HAppS.ACID -> http://happs.org/HAppS/Acid.(hs|lhs) Text.XML.HaXml.Escape -> http://www.cs.york.ac.uk/fp/darcs/HaXml/src/Text/XML/HaXml/Escape.(hs|lhs) Once if finds a module, SearchPath downloads it, put it a directory for that map file, and add that directory to the module search path. However, right now it only knows how to find modules in directories. It would be nice if the URL could point to a Package file rather than a base directory and SearchPath knew how to detect whether it is already installed, install it if not, and import chase all its modules. It would be even nicer if the URL could point to Cabal files and SearchPath knew from the Cabal files what else it needed to download to build the package so that the package author can use the URL of a cabal file in a repository and doesn't need to bother dealing with producing tarballs for each version. I just don't know Cabal well enough to implement this stuff. Does this make sense? -Alex- ______________________________________________________________ S. Alexander Jacobson tel:917-770-6565 http://alexjacobson.com

...

"S. Alexander Jacobson" writes:

(snip)

...
Right now SearchPath only supports module directory base URLs in map files. I'd love for it to support automatic download and build for Cabal file and package URLs as well. I just don't know how to do it. Perhaps you guys can help on this?

You'd just need a way to map module names to package names / versions. One thing you could do is add that kind of lookup to hackage's XML-RPC interface, shouldn't be too hard. Then you can query hackage "what package does this module belong to?"; it'll return a list of package identifiers (meaning package names / versions) and you' could pick among them, download, unpack, and install.

I'd be fine with adding this to the xml-rpc interface if you want. Hopefully Lemmih will speak up if he thinks that's a really bad idea.

Hackage can be gotten here: darcs get http://www.scannedinavian.org/~lemmih/hackage/

with dependencies here: http://www.scannedinavian.org/~lemmih/packages/

peace,

isaac

** CRM114 Whitelisted by: ijones@syntaxpolice.org **

Isaac Jones

11:05 p.m.

New subject: SearchPath

"S. Alexander Jacobson" writes:

...

On Tue, 17 May 2005, Isaac Jones wrote:

...
You'd just need a way to map module names to package names / versions.

Actually, that's the easy part. For example, right now my test map file is:

HAppS http://happs.org *.*.HaXml http://www.cs.york.ac.uk/fp/darcs/HaXml/src

So, any module that starts with HAppS can be found at HAppS.org. And any module with HaXml at the third level of hierarchy can be found in the York HaXml darcs repository. e.g. it does this sort of transformation:

HAppS.ACID -> http://happs.org/HAppS/Acid.(hs|lhs) Text.XML.HaXml.Escape -> http://www.cs.york.ac.uk/fp/darcs/HaXml/src/Text/XML/HaXml/Escape.(hs|lhs)

Once if finds a module, SearchPath downloads it, put it a directory for that map file, and add that directory to the module search path.

However, right now it only knows how to find modules in directories. It would be nice if the URL could point to a Package file rather than a base directory and SearchPath knew how to detect whether it is already installed, install it if not, and import chase all its modules.

Well, the 'hackage' way to do this is to do the reverse lookup I mentioned, download the package, and add the -package flag while you're building, rather than downloading individual modules. It'll look the same from the end-user's perspectivce, though.

...

It would be even nicer if the URL could point to Cabal files and SearchPath knew from the Cabal files what else it needed to download to build the package so that the package author can use the URL of a cabal file in a repository and doesn't need to bother dealing with producing tarballs for each version.

I don't understand. If you use hackage to get from module names to packages and / or .cabal files, then what else do you need? peace, isaac

S. Alexander Jacobson

18 May 18 May

4:14 p.m.

New subject: SearchPath

On Tue, 17 May 2005, Isaac Jones wrote:

...

Well, the 'hackage' way to do this is to do the reverse lookup I mentioned, download the package, and add the -package flag while you're building, rather than downloading individual modules. It'll look the same from the end-user's perspectivce, though.

Yes, my point with SearchPath is to factor hackage down to the bare minimum of being a system for looking up where modules reside. If they reside in a package, then of course it should download the package and install it.

...

I don't understand. If you use hackage to get from module names to packages and / or .cabal files, then what else do you need?

A difference perhaps is that SearchPath integrates grabbing dependencies into the edit/compile/debug cycle rather keeping it separate. Things I'd like to know. 1. Is there a programmatic way to detect what packages are installed or do I have to shell out to ghc-pkg? 2. Is the name in the cabal file guaranteed to match the name exposed by ghc-pkg? 3. Can I access the source of all exposed modules in an installed package to enumerate its external module dependencies and import chase? 4. If you have an unpacked package in a directory, is there a path from the cabal file to the relative paths of each file in the package? (so you can point at a repository rather than a tarball and access it via http). 5. Given that module names are absolute, if you have another way to find external modules required by a package (e.g. SearchPath or perhaps hackage reverse lookup), can you ignore build-depends? -Alex- ______________________________________________________________ S. Alexander Jacobson tel:917-770-6565 http://alexjacobson.com On Tue, 17 May 2005, Isaac Jones wrote:

...

"S. Alexander Jacobson" writes:

...
On Tue, 17 May 2005, Isaac Jones wrote:

...
You'd just need a way to map module names to package names / versions.

Actually, that's the easy part. For example, right now my test map file is:

HAppS http://happs.org *.*.HaXml http://www.cs.york.ac.uk/fp/darcs/HaXml/src

So, any module that starts with HAppS can be found at HAppS.org. And any module with HaXml at the third level of hierarchy can be found in the York HaXml darcs repository. e.g. it does this sort of transformation:

HAppS.ACID -> http://happs.org/HAppS/Acid.(hs|lhs) Text.XML.HaXml.Escape -> http://www.cs.york.ac.uk/fp/darcs/HaXml/src/Text/XML/HaXml/Escape.(hs|lhs)

Once if finds a module, SearchPath downloads it, put it a directory for that map file, and add that directory to the module search path.

However, right now it only knows how to find modules in directories. It would be nice if the URL could point to a Package file rather than a base directory and SearchPath knew how to detect whether it is already installed, install it if not, and import chase all its modules.

Well, the 'hackage' way to do this is to do the reverse lookup I mentioned, download the package, and add the -package flag while you're building, rather than downloading individual modules. It'll look the same from the end-user's perspectivce, though.

...
It would be even nicer if the URL could point to Cabal files and SearchPath knew from the Cabal files what else it needed to download to build the package so that the package author can use the URL of a cabal file in a repository and doesn't need to bother dealing with producing tarballs for each version.

I don't understand. If you use hackage to get from module names to packages and / or .cabal files, then what else do you need?

peace,

isaac

** CRM114 Whitelisted by: ijones@syntaxpolice.org **

Isaac Jones

5:07 p.m.

New subject: SearchPath

"S. Alexander Jacobson" writes: (snip)

...

Things I'd like to know.

1. Is there a programmatic way to detect what packages are installed or do I have to shell out to ghc-pkg?

Yes. Cabal talks to ghc-pkg, you can use the Distribution.Simple.Configure module for this, I think. Look around for a function called "installedPackages" or something.

...

2. Is the name in the cabal file guaranteed to match the name exposed by ghc-pkg?

The package name in the cabal file is the same as the package name for GHC.

...

3. Can I access the source of all exposed modules in an installed package to enumerate its external module dependencies and import chase?

Not the source of binary packages, but the .hi files have this information, I believe.

...

4. If you have an unpacked package in a directory, is there a path from the cabal file to the relative paths of each file in the package? (so you can point at a repository rather than a tarball and access it via http).

Yeah. You can derive it from the module name plus the hs-source-dirs.

...

5. Given that module names are absolute, if you have another way to find external modules required by a package (e.g. SearchPath or perhaps hackage reverse lookup), can you ignore build-depends?

If you have an orthogonal system to do the same thing, of course you can. peace, isaac

S. Alexander Jacobson

5:25 p.m.

New subject: SearchPath

On Wed, 18 May 2005, Isaac Jones wrote:

...

...
3. Can I access the source of all exposed modules in an installed package to enumerate its external module dependencies and import chase?

Not the source of binary packages, but the .hi files have this information, I believe.

Can I access the .hi files of the hidden modules as well? Do you know how to extract import statements from .hi files?

...

...
4. If you have an unpacked package in a directory, is there a path from the cabal file to the relative paths of each file in the package? (so you can point at a repository rather than a tarball and access it via http).

Yeah. You can derive it from the module name plus the hs-source-dirs.

What about other stuff like enclosed C files, makefiles, etc? -Alex- ______________________________________________________________ S. Alexander Jacobson tel:917-770-6565 http://alexjacobson.com

7355

Age (days ago)

7356

Last active (days ago)

List overview

Download

7 comments

3 participants

participants (3)

Isaac Jones
S. Alexander Jacobson
Simon Marlow

RE: hackage, cabal-get, and security

tags

participants (3)