Oleg and I are pleased to announce the release of Takusen 0.8. (Don Stewart did an interim 0.7 release for us a few weeks ago, and added us to Hackage. This release is a tidy-up of some loose ends, and some bug fixes. Hence, I've summarise the changes since the 0.6 release.) Changes since 0.6: - ODBC support. This still has a few gaps (and probably bugs and rough edges) but should be fairly usable. - support for reusable/persistent sessions, so you can hang onto the connection object between invocations of withSession (this was in release 0.6 but omitted from the release notes). - improvements to the Cabal Setup scripts, which should give better experiences for ghc-6.4, ghc-6.6, and ghc-6.8. The (eventual) 1.4 release of Cabal should be able to build our haddock docs, too. - improved UTF8 decoder (marshals directly from buffer). The release bundle: http://hackage.haskell.org/packages/archive/Takusen/0.8/Takusen-0.8.tar.gz The latest code: darcs get http://darcs.haskell.org/takusen Docs: http://darcs.haskell.org/takusen/doc/html/index.html A comprehensive description of API usage can be found in the documentation for module Database.Enumerator (look for the Usage section): http://darcs.haskell.org/takusen/doc/html/Database-Enumerator.html Future plans: - FreeTDS backend (Sybase and MS Sql Server) - support for Blobs and Clobs For those of you unfamiliar with Takusen, here is our HCAR blurb: Takusen is a library for accessing DBMS's. Like HSQL, we support arbitrary SQL statements (currently strings, extensible to anything that can be converted to a string). Takusen's `unique-selling-point' is safety and efficiency. We statically ensure all acquired database resources - such as cursors, connection and statement handles - are released, exactly once, at predictable times. Takusen can avoid loading the whole result set in memory, and so can handle queries returning millions of rows in constant space. Takusen also supports automatic marshalling and unmarshalling of results and query parameters. These benefits come from the design of query result processing around a left-fold enumerator. Currently we fully support ODBC, Oracle, Sqlite, and PostgreSQL.
Changes since 0.6:
- ODBC support. This still has a few gaps (and probably bugs and rough edges) but should be fairly usable.
Doh! I had also meant to make a request: the ODBC code (and Setup.hs configuration) is only tested under Windows (XP). We'd love for people to test on *nix platforms and fill in the missing bits in the Setup script. Alistair
If you want to use your own Setup.hs, you need to specify a Build-Type of Custom, not Simple. Some tools will ignore your Setup.hs if you specify Simple. I guess the message "If possible use 'build-type: Simple'" is a bit confusing.
ross:
If you want to use your own Setup.hs, you need to specify a Build-Type of Custom, not Simple. Some tools will ignore your Setup.hs if you specify Simple. I guess the message "If possible use 'build-type: Simple'" is a bit confusing.
Yes, cabal-setup found this confusing. I wonder if its possible to just use flags for building the different takusen backends, rather than this automagic uber-discovery? -- Don
ross:
If you want to use your own Setup.hs, you need to specify a Build-Type of Custom, not Simple. Some tools will ignore your Setup.hs if you specify Simple. I guess the message "If possible use 'build-type: Simple'" is a bit confusing.
Yes, cabal-setup found this confusing.
I wonder if its possible to just use flags for building the different takusen backends, rather than this automagic uber-discovery?
Mebbe... are you suggesting command-line flags, or cabal flags? I'm certainly open to suggestions for improving the install. Part of the motivation for discovery is that for PostgreSQL, you need to run a program (with various flags) which spits out the locations for include files, libs, etc (presumably because they can vary on different platforms? I'm not really sure why, being a Windows chimp myself). Once you do this, it's not much more effort to extend it to detect other DBMS installations, assuming they have standard layouts (Oracle certainly does). The alternative approach is what HSQL did (and I think HDBC, too), which is to have a base package, and then a separate package for each back-end. I considered this, but I much prefer to have the library available as a single download-build-install process. This makes for a nicer user experience (at the expense of more work for the package maintainer). Isn't this partly the point of cabal's Setup.hs i.e. to accommodate more complex build processes than cabal provides out-of-the-box? Obviously there's some pain, because the cabal API is not stable, but c'est la vie... Alistair
alistair:
ross:
If you want to use your own Setup.hs, you need to specify a Build-Type of Custom, not Simple. Some tools will ignore your Setup.hs if you specify Simple. I guess the message "If possible use 'build-type: Simple'" is a bit confusing.
Yes, cabal-setup found this confusing.
I wonder if its possible to just use flags for building the different takusen backends, rather than this automagic uber-discovery?
Mebbe... are you suggesting command-line flags, or cabal flags? I'm certainly open to suggestions for improving the install.
Part of the motivation for discovery is that for PostgreSQL, you need to run a program (with various flags) which spits out the locations for include files, libs, etc (presumably because they can vary on different platforms? I'm not really sure why, being a Windows chimp myself). Once you do this, it's not much more effort to extend it to detect other DBMS installations, assuming they have standard layouts (Oracle certainly does).
The alternative approach is what HSQL did (and I think HDBC, too), which is to have a base package, and then a separate package for each back-end. I considered this, but I much prefer to have the library available as a single download-build-install process. This makes for a nicer user experience (at the expense of more work for the package maintainer).
Yeah, I really like the single package model. And the Setup.hs does work -- I just can't use cabal 1.2.x or 1.3.x or 1.4.x with it :) So I was imagining cabal flags to select things, but again, this may not work for all backends.
Isn't this partly the point of cabal's Setup.hs i.e. to accommodate more complex build processes than cabal provides out-of-the-box? Obviously there's some pain, because the cabal API is not stable, but c'est la vie...
That's right. It's all some give and take.
On Wed, 2008-03-05 at 15:08 -0800, Don Stewart wrote:
alistair:
ross:
If you want to use your own Setup.hs, you need to specify a Build-Type of Custom, not Simple. Some tools will ignore your Setup.hs if you specify Simple. I guess the message "If possible use 'build-type: Simple'" is a bit confusing.
Yes, cabal-setup found this confusing.
I wonder if its possible to just use flags for building the different takusen backends, rather than this automagic uber-discovery?
Mebbe... are you suggesting command-line flags, or cabal flags? I'm certainly open to suggestions for improving the install.
It would be possible to turn the backends on/off via configuration flags, like: flag odbc library if flag(odbc) exposed-modules: Blah.Backend.ODBC extra-libraries: odbc however that is considered a bit evil, changing the exposed interface based on flags. We did even consider banning that outright (except that the base package does it for the GHc.* modules). It also means that other packages cannot depend on a particular backend, they can only tell users to use certain configure flags. It cannot be automated. What do you do if the user does not have the ODBC or the postgresql C libs installed? Do they have to have all backends installed to use just one? The custom code in Setup.hs to find the flags by calling pg_config etc is perfectly ok. It is unfortunately a bit fragile however but I don't have any better suggestion at the moment. Duncan
Yeah, I really like the single package model. And the Setup.hs does work -- I just can't use cabal 1.2.x or 1.3.x or 1.4.x with it :)
I beg to differ. We have three cabal Setup files in the repo: Setup.hs (for the 1.2 series), Setup-1162.hs, and Setup-114.hs. Setup.hs should work with ghc-6.8 out of the box, Setup-1162.hs should work with ghc-6.6 (once you've upgraded cabal to 1.1.6.2), and Setup-114 I just kept because, well, someone might find it useful. The idea is that you should upgrade cabal to the latest (currently 1.2) and use the Setup.hs, but in case you can't, the older Setup's are hanging around to help. That said, I haven't tested with the latest cabal release, so perhaps it's borken again.
What do you do if the user does not have the ODBC or the postgresql C libs installed? Do they have to have all backends installed to use just one?
No, we do something you might consider fairly hideous. Having established which libraries are installed on the user's system, we then use that list to filter the module list. So if you don't have Oracle or Sqlite installed, the module list is pruned so that these modules are no longer present in the build. This isn't stricly necessary; another approach would be to simlply build all of the modules. This works fine with ghc (which uses gnu ld for linking). However, with ghci (which has a custom module loader and linker, I think), it seems to attempt to link all of the modules in the lib, and this fails badly if you don;t have every back-end DBMS client installed. The only way I could find to fix that was to adjust the build to only include modules that will link successfully (i.e. those for which you have a DBMS client installed). Alistair
On Thu, 2008-03-06 at 08:27 +0000, Alistair Bayley wrote:
Yeah, I really like the single package model. And the Setup.hs does work -- I just can't use cabal 1.2.x or 1.3.x or 1.4.x with it :)
I beg to differ. We have three cabal Setup files in the repo: Setup.hs (for the 1.2 series), Setup-1162.hs, and Setup-114.hs. Setup.hs should work with ghc-6.8 out of the box, Setup-1162.hs should work with ghc-6.6 (once you've upgraded cabal to 1.1.6.2), and Setup-114 I just kept because, well, someone might find it useful. The idea is that you should upgrade cabal to the latest (currently 1.2) and use the Setup.hs, but in case you can't, the older Setup's are hanging around to help.
That said, I haven't tested with the latest cabal release, so perhaps it's borken again.
I've no idea why people are talking about a Cabal 1.4 release, there's been no such thing. :-) BTW, it's not your responsibility to make your Setup.hs compatible with the current development version of Cabal, it's our aim to keep Cabal-1.4 compatible with existing Setup.hs scripts. That said, I did have a look at your Setup.hs and is does use rather a lot of internal Cabal apis.
What do you do if the user does not have the ODBC or the postgresql C libs installed? Do they have to have all backends installed to use just one?
No, we do something you might consider fairly hideous.
Yes, I saw that and was appropriately horrified :-)
Having established which libraries are installed on the user's system, we then use that list to filter the module list. So if you don't have Oracle or Sqlite installed, the module list is pruned so that these modules are no longer present in the build. This isn't stricly necessary; another approach would be to simlply build all of the modules. This works fine with ghc (which uses gnu ld for linking). However, with ghci (which has a custom module loader and linker, I think),
Yes.
it seems to attempt to link all of the modules in the lib, and this fails badly if you don;t have every back-end DBMS client installed.
Right.
The only way I could find to fix that was to adjust the build to only include modules that will link successfully (i.e. those for which you have a DBMS client installed).
Yes, or split into multiple libs. I looked yesterday at a slightly nicer way of doing what you're already doing. We can use flags to turn on/off the various backends. What is tricky is doing that such that the default is chosen automagically. We have a way to set the default value of a flag, but just constants True/False. We don't have any mechanism to adjust flags based on code in Setup.hs. I'll ponder it. Duncan
However, with ghci (which has a custom module loader and linker, I think), it seems to attempt to link all of the modules in the lib, and this fails badly if you don;t have every back-end DBMS client installed.
The only way I could find to fix that was to adjust the build to only include modules that will link successfully (i.e. those for which you have a DBMS client installed).
I looked yesterday at a slightly nicer way of doing what you're already doing. We can use flags to turn on/off the various backends. What is tricky is doing that such that the default is chosen automagically.
To be fair, this is arguably ghci's fault. If the loader/linker in ghci was more like gnu ld, then we could just build all of the modules, and let the linker handle pulling in the optional bits. But I have no idea if ghc HQ plan to fix it. I don't know if there's another use case for this kind of behaviour that would justify adding it to cabal. Alistair
participants (4)
-
Alistair Bayley -
Don Stewart -
Duncan Coutts -
Ross Paterson