The packaging system (or "Cabal"[1] as it's called now) is just one part of an over-all infrastructure that we would like to build some day. We think that building things in layers is important. The bottom layer should be the compile / install infrastructure, so that tools which help with building and installing can be layered on top of that. The tools should do their job, and do it well. The compiler compiles, the build / install system installs, and you can layer a tool on top of that which tracks package databases. For discussion of this, you could read this brief section of the previous proposal (which if you haven't read might provide some useful insights), and some of the old use cases. http://www.haskell.org/libraryInfrastructure/proposal-1.0/x396.html It is very important to me to support tools by Marcus. His packages are actually the most difficult to manage, and the most confusing for an end user. Supporting just tools by Angela would in fact be much easier (except perhaps for implementing things directly in the compilers, as you propose). Fortunately, I think we can support Marcus without making Angela's job harder. If we can step back for a moment, I think you're saying that supporting Marcus isn't that important, or that it's too hard. Maybe we could break up the argument along the lines of: 1) Why we should / shouldn't support Marcus 2) What would / could a system that just supports Angela look like 3) What would a system that supports tools by both Angela and Marcus look like (this proposal is our answer to that). I'm fairly comfortable that our system is appropriate to answer 3. Perhaps I haven't given enough justification for 1? I haven't thought about 2 very much because in my experience, that's the least of my Haskell-related problems, and 3 subsumes 2. Maybe you could give more explicit argument for why 2 is so compelling as to exclude tools by Marcus. (more below...) "S. Alexander Jacobson" writes:

On Mon, 7 Jun 2004, Isaac Jones wrote:

...

...

Any reason we can't simplify Joe's interface to:

ghc --install http://url/of/some.pkg

--or--

hugs --install /path/to/some.pkg

This is quite explicitly something we're trying to support. The original document, and my talk at the Haskell Implementor's Meeting, had some exampled of "layered tools" for the system, including exactly this use. I should probably re-add that to this version of the proposal.

This interface should be _fundamental_. JoeBob*, and Sam are indifferent to how it is implemented. It should just work if the compiler/interpreter is installed correctly.

I would be open to discussing this further if we could get all the compiler authors to agree to implement it. Even if they did, it could still be done using Setup as the interface. This is an orthogonal issue. (And as an aside, I'm not sure if it makes sense to invoke a compiler to install stuff.)

[...sample shell script implementations elided because implementation is irrelevant to this discussion...

They were there to demonstrate how they can be easily implemented once we fix the Setup interface. This demonstrates the power and flexibility of this interface. I'm not sure why you think it's irrelevant.

question: Why are the sample implementations not themselves in Haskell!?...]

It was just an example. Could just as well be implemented in haskell. I know that there's a haskell wrapper for libcurl (which would take the place of wget). I don't know of a haskell unzip library, and I don't know if libcurl is there for windows. (snip)

...

...

The open question is what this setup specification should look like. There is an obvious tradeoff between the interests of Joe and Angela on the one hand and Marcus on the other.

I think our scheme satisfies Joe, Angela, and Marcus. All Joe knows is that he says "haskell-install".

At least in the proposal so far, someone has to deal with "runhugs" or whatever. --install should be standard for every compiler/interpreter just like --package.

Haskell-install deals with the execution of the wrapper script. It does so in a platform-dependent way, and can be made to understand users' compiler preferences or what-have-you.

...

All Angela knows is that she fills out the (declarative) fields required by Distribution.Simple. She doesn't know or care how Joe's operating system implements wget.

Or even IF Joe's os implements wget. Why not implement in Haskell to avoid this issue?

That's perfectly fine. The shell script was just an example.

...

Angela does have this. She can use Distribution.Simple. She's not doing arbitrary or obscure IO. The example we give is very declarative for her.

Sorry, where is the example. 1.2 does not include telling the system that Angela.Internals is private to Data.Set and Data.Bag. 3.4 is empty as is 5.3.

My concern here is in particular that the (declarative) fields are not yet fleshed out even though they are the most important part of this sort of proposal!

The interface is the most important part, and in our proposal, this is not the interface, though you may assert that it should be. We actually have a really good idea of what the fields are, but not what the format is. Fortunately, once we've defined the interface, we can put off being precise about the fields. If you get a chance to take a look at the code, the package description looks like this right now: data PackageDescription = PackageDescription { -- the following are required by all packages: package :: PackageIdentifier, licenese :: License, copyright :: String, maintainer :: String, stability :: String, -- the following are required by the simple build infrastructure only: buildDepends :: [ Dependency ], allModules :: [ String ], mainModules :: [ String ], cSources :: [ FilePath ], exposedModules :: [ String ], extensions :: [ Extension ], extraLibs :: [ String ], includeDirs :: [ FilePath ], includes :: [ FilePath ], options :: [ (CompilerFlavor, [String]) ] }

...

Autoconf and friends implement this. We provide an interface which wraps both autoconf and Distribution.Simple. Sam has been perfectly happy for years using "./configure;make;make install".

So, why are you making Sam do something different for Haskell?

...

OTOH, Marcus needs something more flexible.

Is there some substantive problem with autoconf/make with respect to Haskell such that it needs to be wrapped with Setup.lhs?

No. It's a matter of providing a common interface to users and tools.

If I have a package that is part C, part Python, and part Haskell, why is it reasonable to assume the outer-layer is necessarily Haskell as opposed to e.g. Python's dist-utils?

Packaging depends on your target audience. If the author of such a package doesn't want to implement the Setup structure, there's nothing forcing him to. His users just won't be able to use haskell-install. If they're mostly Haskell users, then this will annoy them. If not, then they won't care.

Or does Marcus need to strategize whether the outer layer is dist-utils or HPS and then have one call the other?

If he cared that much, he could implement both.

* I don't believe Joe and Bob are different people so I am consolidating them as JoeBob. In particular I don't believe that Joe user should have to get someone else's help to do a simple install.

Joe doesn't install new packages, nor does he ask for packages to be installed. He just wants to invoke the haskell compiler. Picture a student in a class where all of the libraries and such are already set up for them. peace, isaac [1] Cabal: Common Architecture for Building Applications and Libraries

Isaac, Bob wants to install libraries, but has only an editor and a haskell compiler/interpreter installed on his machine. Although we can't expect him to build packages created by Marcus (which require a C compiler), there is no reason he shouldn't be able to install packages created by Angela or Peter Packager (new persona!). Peter Packager runs the same OS/hardware as Joe/Bob, but has the tool chain necessary to compile/build a Haskell Package from Marcus' code. He supplies Joe with Haskell Packages that include binaries that run on Joe/Bob's machine. My claim is that the interface between Marcus and Peter should be autoconf/make. Running "./configure;make" should produce a Haskell Package which consists only of some combination of: * haskell source * binaries for Joe's os/hardware (perhaps for multiple platforms?) * documentation/metadata * datafiles Bob can then do e.g. "ghc --install /path/to/package" ("make install" should also have the same effect, but that is really up to Marcus.) Among the consequences of the above: * Cabal becomes completely declarative * It may be possible to warn the user if a package uses IO or unsafeperformIO and to tell the user exactly what binaries are being located where. * Implementation becomes substantially easier

The tools should do their job, and do it well. The compiler compiles, the build / install system installs, and you can layer a tool on top of that which tracks package databases.

From a layering perspective, I am advocating a clean interface between Peter and Bob. It is really a communication protocol that will outlast multiple generations of implementations.

It is very important to me to support tools by Marcus. His packages are actually the most difficult to manage, and the most confusing for an end user.

Agreed. That is why you need something like autoconf/make to build them and we can't expect that of Bob. The question is whether we can do better than autoconf/make in producing Haskell Packages for Bob to install.

Supporting just tools by Angela would in fact be much easier

Good. My claim is that all you need is to support Angela plus binaries.

(except perhaps for implementing things directly in the compilers, as you propose).

I don't mean to require that the compiler implement that interface but rather that you should not need anything more than a Haskell compiler/interpreter to install a package.

Fortunately, I think we can support Marcus without making Angela's job harder.

The issue is not Marcus vs. Angela. It is Marcus vs. Bob. I think you need to support Marcus<->Peter and Peter<->Bob. I don't see how you can do better than autoconf/make for the former and I don't know that there is any support in the current proposal for the later (including binaries).

If we can step back for a moment, I think you're saying that supporting Marcus isn't that important, or that it's too hard. Maybe we could break up the argument along the lines of:

1) Why we should / shouldn't support Marcus 2) What would / could a system that just supports Angela look like 3) What would a system that supports tools by both Angela and Marcus look like (this proposal is our answer to that).

I'm fairly comfortable that our system is appropriate to answer 3.

autoconf/make already support Marcus. The people who need more support are Peter and Bob.

Haskell-install deals with the execution of the wrapper script. It does so in a platform-dependent way, and can be made to understand users' compiler preferences or what-have-you.

The install should not be a script. It should be a declarative specification that allows the installer to warn the user of potentially unsafe things.

We actually have a really good idea of what the fields are, but not what the format is.

It should be Haskell. Angela is guaranteed to understand the syntax and, if you have a Haskell interpreter/compiler, verification is very easy.

Fortunately, once we've defined the interface, we can put off being precise about the fields. If you get a chance to take a look at the code, the package description looks like this right now:

data PackageDescription = PackageDescription { -- the following are required by all packages: package :: PackageIdentifier, licenese :: License, copyright :: String, maintainer :: String, stability :: String,

-- the following are required by the simple build infrastructure only: buildDepends :: [ Dependency ], allModules :: [ String ], mainModules :: [ String ], cSources :: [ FilePath ], exposedModules :: [ String ], extensions :: [ Extension ], extraLibs :: [ String ], includeDirs :: [ FilePath ], includes :: [ FilePath ], options :: [ (CompilerFlavor, [String]) ] }

Looks good, but I don't think you need all the C compilation support stuff. I do think you need a specification for resolving dependency names to actual package files so that installing a package that depends on other packages does not require multiple manual download install processes given a complete archive (CHAN?).

...

Is there some substantive problem with autoconf/make with respect to Haskell such that it needs to be wrapped with Setup.lhs?

No. It's a matter of providing a common interface to users and tools.

They are two distinct needs/users. Marcus delivers an autoconf/make file to Peter. Peter delivers a haskell package to Bob/Joe. -Alex- _________________________________________________________________ S. Alexander Jacobson mailto:me@alexjacobson.com tel:917-770-6565 http://alexjacobson.com

If your Bob the Builder is supposed to be as competent as my Peter the Packager, then you are clearly missing some roles! There are e.g. Win32 installers HDirect and HaskellScript. Who produced them and who is intended to use them? I run GHC on my Windows box and can't easily/reliably get C code to compile and produce DLLs (both because my C is *very* rusty and because I don't have an up-to-date version of all the cygwin mingwin blah blah packages). Yet, I do occasionally download and install Haskell libraries? Am I Bob, Peter, Sam, Joe, or perhaps someone new? -Alex- _________________________________________________________________ S. Alexander Jacobson mailto:me@alexjacobson.com tel:917-770-6565 http://alexjacobson.com On Tue, 8 Jun 2004, Malcolm Wallace wrote:

"S. Alexander Jacobson" writes:

...

Bob wants to install libraries, but has only an editor and a haskell compiler/interpreter installed on his machine. Although we can't expect him to build packages created by Marcus (which require a C compiler), there is no reason he shouldn't be able to install packages created by Angela or Peter Packager (new persona!).

It seems to me that Bob the Builder does not deserve the name if he doesn't have the facilities to build a random package from source! If Bob is really that naive then his name is Joe User, who only wants to use the package, not build it.

Your Peter Packager seems to have exactly the attributes we intended Bob the Builder to have. So you have conflated Joe and Bob into a single persona, necessitating the introduction of Peter, but in our minds Bob and Peter are the same persona, quite separate from Joe.

Regards, Malcolm _______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries