Re: new Library Infrastructure spec.

Hi Arie, Arie Peterson writes:

At 18:12 2004-06-03 -0400, Isaac Jones wrote:

...

We're trying to give the end users (and tools that may be layered on top of the packaging system) a consistent interface to packages whose needs and underlying implementations are very different. (Think of the different kinds of needs between HUnit and wxHaskell).

Now Distribution.Simple is a particular implementation of this interface which is meant for very simple libraries and tools.

What you propose to provide is a secure infrastructure for building and installing very simple libraries and tools, but not providing a common interface across diverse libraries and tools. I feel that this is a completely different goal. (I also think that if this is what's desired, the user is better off using a packaging system that has more guarantees, and not installing code from untrusted sources at all (see my work in "apt-secure" btw) but that's neither here nor there.)

I do not understand why a strictly defined data structure would not be a common interface, and why a (for ignorant windows users like me, at least) obscure and opaque makefile would.

I'm afraid you must misunderstand something. A makefile is not the interface. The interface is the Setup script and the arguments to it (install, build, etc).

Do you mean by 'interface' the way in which a user invokes the installation? I would like to be able to choose whether I type "make install" myself,

I don't understand; the "choice" of how to install the system is not made by you (Joe, Bob, or Sam), the installer, but rather by the author (Angela or Marcus).

let a linux package manager handle that for me, or perhaps use some customised homemade install manager to give me exactly the information and options I want.

This doesn't replace RPM or Debian-type package management; it facilitates it (for Ronald, Donald, and Willie) . For instance, the "common debian build system" can be designed to wrap the Setup interface; making it very easy to make Debian packages for Haskell libraries and tools. In particular, all a packager has to do for a package that conforms to the interface, is write a "control" file that says "build this like a standard haskell package". I already implemented a prototype of this, and it works nicely. OTOH, for those of you who aren't lucky enough to have a nice package manager on your operating system, the Setup script will provide a consistent interface for installing packages, and in the future perhaps for wrapped tools for downloading and installing packages from a remote database, such as CPAN.

Alex' idea of an installation data structure suits that wish perfectly. As mentioned before, it would be particularly nice to know what files/directories are to be modified or added.

Package managers such as dpkg typically handle this kind of thing. If you use one of those, you'll still have this ability for packages installed via dpkg. However, providing such a facility is not possible for packages by Marcus. This kind of security is properly handled on the operating system level. If you are downloading random code from the web and installing it outside your package manager, haskell packages are the least of your problems.

...

Distribution.Simple provides some mechanisms for making Angela's life easier, especially with very simple libraries. Now say Angela has a library that uses Distribution.Simple, but she's altered it to utilize the Happy preprocessor, and (let's say) Distribution.Simple doesn't know anything about Happy. Now Angela can add a "system" call to her Setup file to preprocess what she needs to, and continue using the rest of the Distribution.Simple mechanism she was using before.

In the StaticSetup scheme, Angela is out of luck. She can't change the behavior of StaticSetup to do the right thing, so she has to get rid of her build system altogether and start from scratch with makefiles or something. This is what we want to avoid.

If the preprocessor is to be executed by the user,

Which user are you referring to? We gave them all names :)

then this is just a dependency and can be listed as such in the install structure. If the preprocessor is to be executed by Angela, then she does so happily (sorry ;-)) and the user need not know that a preprocessor was used.

I'm confused by this. Maybe if you told me which user you're referring to I could explain it.

Let me suggest a different approach.

We could define a Setup/Installation monad, together with some some primitives (display a message, ask for user input, move files, check dependencies, request installation of other packages, etc.). One special primitive might be 'IO', containing an arbitrary IO operation, to accommodate installation actions that cannot be defined in terms of the given primitives.

A package description/installation might consist of a haskell data structure, holding some basic properties of the package and a Setup () structure.

If the installer system, provided by Angela, can perform arbitrary IO actions, then you have lost the property of knowing which files it installs.

This package structure may be distributed: 1. as haskell source code alone, to allow users to install the package with their own customised installer or a default ghc installer;

How does the user customize an installer for darcs?

2. in source form with an installer that the user may or may not use; 3. compiled by the author with her installer to produce a binary executable installer.

Our system covers 2 and 3.

Notice that an 'installer' in this context is a haskell program, typically run with hugs or ghci (unless dynamic linking will be widely supported), that merely has a Setup () as a parameter and interprets

I'm confused by who is providing this installer, and how this Setup structure gets provided to an executable.

this structure on the fly, in a way that the author of the installer thinks best: install the package in the right directory, do or do not download necessary libraries automatically, do or do not ask the user to authorise the IO parts,

Why ask the user if they want to authorize the IO parts? They will have to read the code to find out what the IO actions do.

register the program with the compiler of choice, etc.. The installer might get settings from a conf file or bother the user with every detail - that is up to the author of the installer.

I feel somewhat awkward to suggest this wild idea, because the proposal seems well-thought out and quite mature. So please convince me that this approach is useless :-).

I'm afraid I do not completely understand your approach. Most especially, I'm confused about what it buys you that our approach does not. I do suspect that we're losing support for Marcus, which is not acceptable. Further, I think that once you completely understand our approach, you'll like it. Please provide suggestions for the proposal for bits which are unclear. peace, isaac