decisions on building infrastructure

Greetings, Most of the discussion about the "library infrastructure" has been about whether to start with a build system or a distribution system. The solution to that discussion, in my mind, is to come at it from both sides and meet in the middle. Peter Simons will be working on a proposal for distribution, and we'll be working together to integrate some kind of package / build / install / distribute mechanism. With that in mind, we need to make some decisions about what the build infrastructure will look like. To recap, I propose that we have 1) a "default" makefile system. Alastair Reid is working on one based on fptools' makefiles. We can provide a tool to generate a skeleton makefile system for new libraries. 2) an abstraction layer, perhaps like "distutils" for python combined with Debian's deiban/rules file which will allow us to build layered tools which will: - Make it so that 3rd party library authors don't _have_ to change their build system - Make it easier for packagers to make Debian / Redhat / FreeBSD packages, - Make it easier for 3rd parties to distribute their libraries on their home pages (probably in a source tarball) - Make it easier for end users to download, install, and keep up to date 3rd party libraries My proposal is on the wiki (see URL below). Does anyone have comments about the proposal? Does anyone think that something similar to what Python has is a bad idea? http://www.haskell.org/hawiki/LibraryInfrastructure peace, isaac

On Wed, Jun 04, 2003 at 12:16:19PM -0400, Isaac Jones wrote:
With that in mind, we need to make some decisions about what the build infrastructure will look like. To recap, I propose that we have
1) a "default" makefile system. Alastair Reid is working on one based on fptools' makefiles. We can provide a tool to generate a skeleton makefile system for new libraries.
2) an abstraction layer, perhaps like "distutils" for python combined with Debian's deiban/rules file which will allow us to build layered tools which will:
- Make it so that 3rd party library authors don't _have_ to change their build system - Make it easier for packagers to make Debian / Redhat / FreeBSD packages, - Make it easier for 3rd parties to distribute their libraries on their home pages (probably in a source tarball) - Make it easier for end users to download, install, and keep up to date 3rd party libraries
Sorry to be prosaic, but I would prefer to start at the bottom level, because there's a pressing need for a build system, and ideas there can be tested with code. I think an important requirement is that most of the build system should be shared, either distributed separately or bundled with the compilers, or a bit of each, but definitely not duplicated in each library. The library author should supply the absolute minimum information that is specific to the library, preferably in a declarative form. I'm not too worried about 3rd party library authors being forced to change their build systems; nobody has one that works with all 3 compilers (except maybe Yale). And the information required by the common build system should be minimal.

Ross Patterson wrote:
I'm not too worried about 3rd party library authors being forced to change their build systems; nobody has one that works with all 3 compilers (except maybe Yale). And the information required by the common build system should be minimal.
Yes, we _almost_ have one, based on gnumake. But the complexity of making it work with Haskell systems as different as Hugs and GHC, on platforms as different as Unix/Linux and Windows, while providing flexible support for all kinds of pre-processors, submodules written in languages other than Haskell, and so on, became, in the end, quite daunting, to say the least. So we're currently out of steam. The last stumbling block, I believe, was something utterly silly like spaces in path names (quite common under Windows), that caused some carefully crafted snippets of shell code to fall over. Sigh.
Sorry to be prosaic, but I would prefer to start at the bottom level, because there's a pressing need for a build system, and ideas there can be tested with code.
That's probably a good way to go. At least that has been our approach. (Our intention was (we've got parts of it working) to add functionality for distribution and packaging simply by adding extra make targets.)
I think an important requirement is that most of the build system should be shared, either distributed separately or bundled with the compilers, or a bit of each, but definitely not duplicated in each library. The library author should supply the absolute minimum information that is specific to the library, preferably in a declarative form.
In our case, we didn't want to require the end user to fetch a build system separately. So, as always seem to be the case with make files, we intended our build system to be distributed with our libraries (and applications, it supports both). But the library (or application) writer only has to supply some more or less declarative specifications in a few key places: most of the build system can be copied verbatim. Somewhat crude (as make based systems tend to be), but it almost works for us. And with a little extra effort, I'm fairly confident we can make it work as intended also on Windows. Best regards, /Henrik -- Henrik Nilsson Yale University Department of Computer Science nilsson@cs.yale.edu

Ross Paterson
On Wed, Jun 04, 2003 at 12:16:19PM -0400, Isaac Jones wrote:
With that in mind, we need to make some decisions about what the build infrastructure will look like. To recap, I propose that we have
1) a "default" makefile system. Alastair Reid is working on one based on fptools' makefiles. We can provide a tool to generate a skeleton makefile system for new libraries.
2) an abstraction layer, perhaps like "distutils" for python combined with Debian's deiban/rules file which will allow us to build layered tools which will:
- Make it so that 3rd party library authors don't _have_ to change their build system - Make it easier for packagers to make Debian / Redhat / FreeBSD packages, - Make it easier for 3rd parties to distribute their libraries on their home pages (probably in a source tarball) - Make it easier for end users to download, install, and keep up to date 3rd party libraries
Sorry to be prosaic, but I would prefer to start at the bottom level, because there's a pressing need for a build system, and ideas there can be tested with code. I think an important requirement is that most of the build system should be shared, either distributed separately or bundled with the compilers, or a bit of each, but definitely not duplicated in each library. The library author should supply the absolute minimum information that is specific to the library, preferably in a declarative form.
I'm not too worried about 3rd party library authors being forced to change their build systems; nobody has one that works with all 3 compilers (except maybe Yale). And the information required by the common build system should be minimal.
Being a 3rd party library author, I am worried. Gtk+HS, for example, has a quite sophisticated build system. Adapting that to anything else would be quite some work. IMHO it is nice to have a build infrastructure that people can fall back on if they haven't got anything else, but to prescribe anything won't work. Cheers, Manuel

Manuel:
IMHO it is nice to have a build infrastructure that people can fall back on if they haven't got anything else, but to prescribe anything won't work.
I strongly agree with this. That's the beauty of Isaac's idea (adapted from python or debian or ...) of putting a script on top of it all. If it's sitting on top of an fptools make system, you use one version of the script; if sitting on top of the gtk+hs build system you use a slightly different version. (In both cases, the script should be fairly trivial because the makefiles do all the work.)
Being a 3rd party library author, I am worried. Gtk+HS, for example, has a quite sophisticated build system. Adapting that to anything else would be quite some work.
This reminds me: can you recommend some make rules for use with c2hs so that I can add them to the fptools infrastructure. -- Alastair

Ross:
I think an important requirement is that most of the build system should be shared, either distributed separately or bundled with the compilers, or a bit of each, but definitely not duplicated in each library. The library author should supply the absolute minimum information that is specific to the library, preferably in a declarative form.
This is what I'm aiming for with the fptools infrastructure (see fptools/libraries/X11 in CVS) I'm successfully building several libraries off a single fptools CVS copy. Some of the remaining issues are: 1) I'd like source distributions to be reasonably complete so, for now, I plan to include a complete copy of fptools in each tarfile. Ideally, this would become a separate package (probably called fpconfig) but this requires us to choose a standard place where fpconfig is installed so that the makefiles can refer to it. (Hmmm, or fpconfig includes a program /usr/bin/fpconfig-info which returns the necessary information?) 2) You need to provide your own copies of: $top/Makefile $top/mk/version.mk $top/mk/boilerplate.mk $top/mk/target.mk Ideally, there'd be just one or two files (I think this can be achieved fairly easily. I'm at the stage where I'm so glad anything works at all that I lost the energy to push further.) 3) The configure script is in fptools. This means a large chunk of boilerplate is shared by many people but also means that there's no ability to customize it by: - adding a test for a new feature - specifying what version of a given tool is needed (and having configure fail if it's not present). It seems like the fix will be to have a very, very minimal configure script in every project which #includes ${fpconfig}/configure.in -- Alastair

On Thu, Jun 05, 2003 at 10:40:34AM +0100, Alastair Reid wrote:
This is what I'm aiming for with the fptools infrastructure (see fptools/libraries/X11 in CVS)
I'm successfully building several libraries off a single fptools CVS copy.
Some of the remaining issues are:
1) I'd like source distributions to be reasonably complete so, for now, I plan to include a complete copy of fptools in each tarfile. Ideally, this would become a separate package (probably called fpconfig) but this requires us to choose a standard place where fpconfig is installed so that the makefiles can refer to it. (Hmmm, or fpconfig includes a program /usr/bin/fpconfig-info which returns the necessary information?)
2) You need to provide your own copies of:
$top/Makefile $top/mk/version.mk $top/mk/boilerplate.mk $top/mk/target.mk
Ideally, there'd be just one or two files (I think this can be achieved fairly easily. I'm at the stage where I'm so glad anything works at all that I lost the energy to push further.)
That's the sort of thing I had in mind. Ideally the author would write a minimal definitions.mk containing only make variable settings specific to the library, and the build system would supply a script that invoked make on a generic Makefile that included definitions.mk in the middle.
participants (5)
-
Alastair Reid
-
Isaac Jones
-
Manuel M T Chakravarty
-
nilsson@cs.yale.edu
-
Ross Paterson