Cool C libraries we could have on hackage server
Hi, I just submitted bindings to libusb, and realized I should comment pkgconfig-depends in .cabal since libusb-1.0 is not installed on the server, or cabal would refuse building it. There are many libraries we probably would like to have bindings for, but we need to have them on hackage to do that. What if we generate a wish list of really trusted and important libraries we would like to use, and then test if they build on hackage, and install in /usr/local those that do? I have a few sugestions: http://www.gnu.org/software/gsl http://www.hdfgroup.org/HDF5 http://libusb.sourceforge.net http://sourceware.org/libffi http://directory.fsf.org/project/mailutils Best, Maurício
Maurício wrote:
There are many libraries we probably would like to have bindings for, but we need to have them on hackage to do that.
I really don't think that is true. Just to pick one that I am aware of: http://hackage.haskell.org/cgi-bin/hackage-scripts/package/hsndfile This relies on libsndfile and was successfully built on the hackage server.
What if we generate a wish list of really trusted and important libraries we would like to use, and then test if they build on hackage, and install in /usr/local those that do?
Thats probably a really bad idea on Debian or Debian derived distributions, where Haskell bindings to these libraries should really link against the versions of the libraries installed by the package manager. On Linux (probably true for Unix in general), Haskell bindings to C libraries should use pkg-config in the build process to find the location of the required libraries and headers. Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/
There are many libraries we probably would like to have bindings for, but we need to have them on hackage to do that.
I really don't think that is true.
Just to pick one that I am aware of:
http://hackage.haskell.org/cgi-bin/hackage-scripts/package/hsndfile
hsndfile actually carries the one file it needs (sndfile.h) with itself. Linking problems are delegated to those using the library, something I would not like to do in a few packages where I do want to release the user from these problems (see bindings-libusb, bindings-sqlite3). It doesn't ensure much compatibility between versions of the base library, although I don't know libsnd enough to say if this is a problem to this particular package.
What if we generate a wish list of really trusted and important libraries we would like to use, and then test if they build on hackage, and install in /usr/local those that do?
Thats probably a really bad idea on Debian or Debian derived distributions, where Haskell bindings to these libraries should really link against the versions of the libraries installed by the package manager.
My impression was that the server was kept at an old version, maybe for reliability issues, since I did 'apt-cache' and didn't find libusb-1.0, only the old libusb-0.1, but I understand nothing of debian or good practices on server management. I just thought that if upgrading the server was an issue, let us build packages outside its package management tool.
On Linux (probably true for Unix in general), Haskell bindings to C libraries should use pkg-config in the build process to find the location of the required libraries and headers.
Exactly! You need those libraries installed so that cabal accepts building your pkg-config dependent package. I do want to release an installer with a set of libraries in my bindings-* packages with pkg-config for Windows too. Thanks for your comments, Maurício
Maurício wrote:
hsndfile actually carries the one file it needs (sndfile.h) with itself.
On Unix I would not consider that best practice but it is understandable if you want windows compatibility.
It doesn't ensure much compatibility between versions of the base library, although I don't know libsnd enough to say if this is a problem to this particular package.
As the upstream author of libsndfile I know that an early header file with a later version of the library is ok as long as you stick with version 1.0.x of the library. Later header file with an earlier version of the library (with the 1.0.x series) can in theory, lead to problems that require runtime checks of the libsndfile version to work around.
My impression was that the server was kept at an old version, maybe for reliability issues, since I did 'apt-cache' and didn't find libusb-1.0, only the old libusb-0.1, but I understand nothing of debian or good practices on server management. I just thought that if upgrading the server was an issue, let us build packages outside its package management tool.
In my day job I am responsible for a quite sophisticated build system built on top of Debian and producing Debian packages as its main output. This autobuilder builds everything in a minimal chroot environment satisfies all the build dependancies, builds the package, uploads it to our own debian respoitory before trying to install it (from the repositry) in a second chroot environment thereby ensuring runtime dependancies are also working. This system (a bit of a hack and lacks a bunch of generality and configurability) works pretty damn well.
Exactly! You need those libraries installed so that cabal accepts building your pkg-config dependent package.
What I was getting at with my description above (and knowing nothing about how hackage actually works) is that you should not have to install the library on the server itself, but that the package should be built in a minimal chroot and the build procedure should pull in the required dependancies.
I do want to release an installer with a set of libraries in my bindings-* packages with pkg-config for Windows too.
A system like I described above should work on any Unix-like system. Windows (and to some extent Mac OSX), because it lacks : - A standard packaging tool. - A standard location for header files. - A standard location for library binaries. - A standard set of tools for asking the system what is installed. makes building code in high level languages that link to C libraries a royal PITA. Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/
On Fri, 2009-06-05 at 10:03 +1000, Erik de Castro Lopo wrote:
Maurício wrote:
There are many libraries we probably would like to have bindings for, but we need to have them on hackage to do that.
I really don't think that is true.
Just to pick one that I am aware of:
http://hackage.haskell.org/cgi-bin/hackage-scripts/package/hsndfile
This relies on libsndfile and was successfully built on the hackage server.
This is quite by accident and not by design.
From the build log: Using Cabal-1.6.0.1 compiled by ghc-6.8
It will fail as of Cabal-1.6.0.2+ since configure now checks that required header files and libs actually exist on the machine. Even without the check, it'd fail in future if we move to using shared libs by default rather than static libs. This issue of what is available on the current single hackage build client shows the flaws in the current design. There should not be a single hackage build client, there should be many. That way it doesn't matter if not everyone has a particular C lib installed. We have such a system 3/4s implemented but it is stalled due to lack of volunteer developer time. Any volunteers to pick up the development of the new happstack-base hackage server? It's nearly at feature parity with the current server. It can already accept upload of build reports from cabal-install clients. It's just lacking docs upload/download and proper management of authentication. http://code.haskell.org/hackage-server/ Duncan
Duncan Coutts wrote:
Any volunteers to pick up the development of the new happstack-base hackage server? It's nearly at feature parity with the current server. It can already accept upload of build reports from cabal-install clients. It's just lacking docs upload/download and proper management of authentication.
Interesting! I took a look, and it was good to see larger example of a real-world happs app. I've sent you a patch that makes it run with happstack 0.2.
Any volunteers to pick up the development of the new happstack-base hackage server? It's nearly at feature parity with the current server. It can already accept upload of build reports from cabal-install clients. It's just lacking docs upload/download and proper management of authentication.
This is way cool! I think many capable people would be interested, specially: those trying to build darcs hosting, like patch-tag, teachers on universities who want local hackage for their haskell students, haskell developers who want hackage on their personal web site. It really deserves a subject on its own :) Have you posted on haskell-cafe announcing the need for help, what is missing, and what users can get after it's done? Maurício
If I were involved in the development of cabal (the tool) then I'd be
resisting the slippery slope towards becoming a general package
manager. The line should be drawn somewhere, and I personally think
that cabal (the tool) should _only_ be concerned with building Haskell
packages, if a Haskell package depends on a C library then that
library has to be installed prior to the invocation of cabal (the
tool), and in a manner that fits the user's system (e.g. using apt-get
on Debian).
Just my 2p...
/M
On Fri, Jun 5, 2009 at 12:53 AM, Maurício
Hi,
I just submitted bindings to libusb, and realized I should comment pkgconfig-depends in .cabal since libusb-1.0 is not installed on the server, or cabal would refuse building it.
There are many libraries we probably would like to have bindings for, but we need to have them on hackage to do that. What if we generate a wish list of really trusted and important libraries we would like to use, and then test if they build on hackage, and install in /usr/local those that do?
I have a few sugestions:
http://www.gnu.org/software/gsl
http://directory.fsf.org/project/mailutils
Best, Maurício
_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries
-- Magnus Therning (OpenPGP: 0xAB4DFBA4) magnus＠therning．org Jabber: magnus＠therning．org http://therning.org/magnus identi.ca|twitter: magthe
Am Freitag, 5. Juni 2009 01:53:30 schrieb Maurício:
I just submitted bindings to libusb, and realized I should comment pkgconfig-depends in .cabal since libusb-1.0 is not installed on the server, or cabal would refuse building it.
That's a general problem for many bindings if you directly use plain "foreign import".
There are many libraries we probably would like to have bindings for, but we need to have them on hackage to do that. What if we generate a wish list of really trusted and important libraries we would like to use, and then test if they build on hackage, and install in /usr/local those that do?
This approach simply doesn't scale, or at least involves *much* work and build infrastructure: Some Linux distros e.g. build packages by creating a build environment from scratch(!) in a chroot environment, containing only those dependencies which are stated by the package description, but this is not something we should expect from Hackage IMHO. After my long experience with the OpenGL/GLUT/OpenAL/ALUT packages, I think that one should forget about plain 'foreign import' completely and lookup API entries dynamically at runtime, using 'foreign import "wrapper"' plus the usual CAF trick for memoization of the entry (see e.g. the OpenGLRaw package). This completely removes the necessity for any libraries or headers on the build machine, with the f.i.w. effectively acting like prototypes in a .h file. I am well aware that this introduces a few minor portability issues, but this is a price I'll happily pay for being dependency-free during building, and most issues have an easy workaround: Platforms which e.g. don't support dynamic linking can easily be handled in a centralized way, too, by emulating dlsym and friends, with no changes at all on the Haskell side. On such platform you have more build dependencies then, but there is no way around this. Luckily, all major platforms don't have these issues. One note about changes in the C API: Even plain 'foreign import' won't help you when the C API changes, you'll have to revert to autoconf or an equivalent technology then, something most people don't like, see the last decade of comments in various Haskell mailing lists. Cheers, S.
participants (6)
-
Duncan Coutts -
Erik de Castro Lopo -
Magnus Therning -
Maurício -
Simon Michael -
Sven Panne