On Sun, Aug 1, 2010 at 9:10 AM, David Anderson wrote:

Congrats on the release.

Just one humble suggestion: your email assumes that the reader already knows what Takusen is. Reading the email, all I can infer is that it has something to do with databases, because of the ODBC reference. The only link in the email also does nothing to explain, since it just links to a ball of code. The README there also assumes that I already know that I want Takusen, and so doesn't bother to explain what it does, only how to use it.

Sorry about that. The description from hackage: http://hackage.haskell.org/package/Takusen Takusen: Database library with left-fold interface, for PostgreSQL, Oracle, SQLite, ODBC. Takusen is a DBMS access library. Like HSQL and HDBC, we support arbitrary SQL statements (currently strings, extensible to anything that can be converted to a string). Takusen's unique selling point is safety and efficiency. We statically ensure all acquired database resources - such as cursors, connections, and statement handles - are released, exactly once, at predictable times. Takusen can avoid loading the whole result set in memory, and so can handle queries returning millions of rows in constant space. Takusen also supports automatic marshalling and unmarshalling of results and query parameters. These benefits come from the design of query result processing around a left-fold enumerator. Currently we fully support ODBC, Oracle, Sqlite, and PostgreSQL.

Ideally, release announcements should always include a 1-sentence executive summary of what the project is, before heading on to what is new. Say, "The Takusen team would like to announce the latest release of Takusen, 0.8.6. Takusen is . This is primarily a bugfix release..."

Thanks, I'll remember that. Jason

aditya.siram:

Why are the Takusen module links on Hackage dead?

Hmm. The links look fine: http://hackage.haskell.org/package/Takusen-0.8.6

this opportunity to request a Takusen tutorial and to thank you for this

A reasonable guess (I think, anyway): the reason is because support for ODBC, Oracle, Postgres etc isn't compiled in by default. You have to specify it with a flag with cabal install to get support for those things. But the reason they show up in API docs I would guess is because Haddock doesn't check constraints on what modules are 'exposed' given certain flags in the cabal file, it just looks over every 'exposed' module no matter what. In this case, it's not really a huge burden because I believe those modules each have almost identical interfaces, in particular they specify a 'connect' like function to get a database handle, and that's about all. The rest is common code under Database.Enumerator. On Sun, Aug 1, 2010 at 1:49 PM, aditya siram wrote:

I meant the links to the API docs. -deech

[1] http://hackage.haskell.org/packages/archive/Takusen/0.8.6/doc/html/Database-...

On Sun, Aug 1, 2010 at 1:46 PM, Don Stewart wrote:

...

aditya.siram:

...

Why are the Takusen module links on Hackage dead?

Hmm.  The links look fine:

   http://hackage.haskell.org/package/Takusen-0.8.6

...

this opportunity to request a Takusen tutorial and to thank you for this

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

-- - Austin

On Sun, Aug 1, 2010 at 11:44 AM, aditya siram wrote:

Why are the Takusen module links on Hackage dead? I would also like to take this opportunity to request a Takusen tutorial and to thank you for this innovative library.

http://blog.codersbase.com/2010/08/takusen-tutorial-part-1-hello-takusen.htm... There you go for a starter tutorial. For some reason the formatting on the published page is quite different compared with the interactive editor, and I got sick of fighting with it. I hope you can still read it well enough to get something out of it. Feed back welcome! I hope to write more in that series. Do you have any suggestions for a simple database application I could develop in the tutorial series? Thanks, Jason

On Mon, Aug 2, 2010 at 5:48 AM, Yitzchak Gale wrote:

Jason Dagit wrote:

...

There you go for a starter tutorial.

Thanks Jason, that's a great start. It already goes a long way towards making Takusen more accessible.

I noticed that QuickCheck is still a dependency. That's not a good idea. It creates a lot of dependency problems, and most of the time the tests aren't needed anyway.

You're right. I may have been able to avoid it by disabling the tests when I installed it. I'll have to look at it more.

It would also be nice if the various backends could be broken out into separate packages too, like for the other database packages. But I understand that might be a bit more work.

This is a definite goal! Thanks, Jason

On 8/1/10 12:12 PM, austin seipp wrote:

Hi Jason,

I've had my eye on the 'Takusen' approach for a while. In particular I think it's a wonderful idea to use the left-fold based interface. Takusen is also well supported and pretty stable, having been around for a while.

I agree; in fact, I use it for all of my database needs.

* It would be nice if we could make it depend on nicer libraries instead of rolling its own stuff - for example, we have Lato's excellent iteratee package, and Bas van Dijk has written a (IMO woefully underappreciated!) 'regions' package with encapsulate the lightweight monadic regions idea Oleg proposed. Of course, due to design, neither of these may work properly for Takusen's case, and if they did they would very likely facilitate API changes, but it's something I've been thinking about in the name of making the library smaller and more lightweight.

Making the library depend on more external libraries will actually make it more "heavyweight" since it will require the user to install more libraries just in order to use Takusen. I'm not saying that this is a bad thing, but I wouldn't automatically count it as a disadvantage that one can install Takusen without requiring lots of other libraries to be installed first. It looks to me like the iteratee package wouldn't be a good fit for Takusen because (as far I understand it) it is designed for the use case of reading raw data where in particular the "chunks" might not be aligned with the records being processed, whereas Takusen is designed for reading in very structured data. The regions package might work, although there are problems with the way that it handles exceptions that has been discussed previously on this list. Finally, a disadvantage of changing Takusen to use these kinds of external libraries is that it could actually make it *harder* to understand how to use it, since first the user would have to understand the concepts involved in the external libraries. Again, I'm not saying that any of these issues automatically make it a bad idea to modify Takusen to use external libraries, just that the current approach works well and is (in my opinion) already relatively simple and straightforward, and it would be unfortunate if this were lost without a clear benefit. Cheers, Greg

Using the generous resources of community.haskell.org I've created a mailing list for takusen discussions. I encourage interested parties to join that list and maybe move the takusen design discussion there: http://projects.haskell.org/cgi-bin/mailman/listinfo/takusen I've added the list in the CC. I've also created a trac instance for wiki/bug tracking: http://trac.haskell.org/takusen/report My comments are below. On Sun, Aug 1, 2010 at 12:55 PM, Alistair Bayley wrote:

At the risk of seeming a bit defensive, I'll respond to some of these points...

...

Despite this, it seems to have a couple faults: * Few tutorials, aside from the Haddocks in Database.Enumerator

True. I put a bit of effort in to writing the docs in Database.Enumerator as a sort of tutorial, but it doesn't work as well as a properly written tutorial. Perhaps this would be better placed on the wiki, but it's quite a bit of work, assuming you would prefer something more than a simple cut-n-paste from the generated docs.

More tutorials is a great idea. Perhaps I can write a few simple 'how to get started' examples and post them on my blog. Although, the wiki is a nice place too. Any preference? If I post it on my blog it might be easier to control the formatting and voice, but if it's on the wiki other can update it and it's in a central location.

...

* Very crufty old code in some spots (I see lots of references to GHC 6.6 and the 'extensible exceptions' changes in the cabal file among other places, which I believe we're all well beyond now. There also seems to be random tidbits that could be removed in favor of a library/removed because they're not necessary.) This should IMO all be removed under the argument "Get a more recent GHC" although people may disagree here.

Maybe... We've put some effort into supporting older versions of ghc, mainly because quite a few distributions have quite long update cycles. If you're stuck in an environment (some unis, some employers?) where you are only allowed tools from the last stable distribution, you may well be many releases behind current ghc. If everyone agrees that 6.8 should be the oldest ghc we should test and support, then that does make things a little simpler. Are there any distros still shipping with ghc-6.6?

This same issues comes up fairly often on the darcs-users mailing list. My understanding of the way things are handled there, is that if there is ever a good reason to drop support for a version of GHC then the person who wants to drop support is supposed to look and see what is in Debian stable. If debian stable is still covered after the proposed changes then they are accepted automatically. If Debian stable would not be covered, then there is a discussion to reach consensus. The reasons it works are: a) support is dropped lazily when there is a real demand to do so, instead of artificially; b) letting people discuss it case-by-case allows the decision to be made with full context.

BTW, I thought extensible-exceptions first shipped with 6.10. I don't think everyone is off 6.8 yet, so we'd want to keep that cabal switch in for a little longer if we intend to support 6.8.

...

* It would be nice if we could make it depend on nicer libraries instead of rolling its own stuff - for example, we have Lato's excellent iteratee package, and Bas van Dijk has written a (IMO woefully underappreciated!) 'regions' package with encapsulate the lightweight monadic regions idea Oleg proposed. Of course, due to design, neither of these may work properly for Takusen's case, and if they did they would very likely facilitate API changes, but it's something I've been thinking about in the name of making the library smaller and more lightweight.

iteratee and regions were both released well after Takusen was written. I think iteratee is just the same idea generalised to other types (Binary, ByteStrings, IO, etc). Perhaps it would help to use it and standardise the interface, but I haven't looked into it at all. It looks like it might be a considerable amount of work.

I think that iteratees are a wonderful concept and I like them, but...I'm not terribly fond of the iteratee package. I find that with it's full generality that it is somewhat difficult to program with. I've actually found it easier to 'roll your own' when I need iteratees. My preference would be to switch to some later iteration of the iteratee design. I believe John Lato is currently working on a new API.

Using regions might be more feasible (i.e. easier), but again I haven't considered it. Takusen's implementation also pre-dates regions considerably, and it works well for us as is. The actual implementation is very little code.

I haven't looked at the regions package either, but it certainly sounds like a good idea to switch to it.

That is, in both cases I don't see a big benefit from switching over to these libraries, considering the amount of work I think it would take (esp for iteratee). There are plenty of other things on the to-do list which I think should take higher priority e.g.

Refactoring to use more libraries is a good long term goal and once it's done, it may make other development easier/quicker and more accessible to people who are not core takusen contributors. I agree that it's not a high priority for me though. The takusen code that uses those concepts is direct enough that the embedded implementations are fine with me, for now.

- splitting Takusen into multiple packages: a core interface, and then separate packages for each backend.

In my opinion, this should be the highest priority refactor. I believe it brings the most benefit to takusen and to other competing database libraries. I don't think that each database library on hackage needs to reimpement the FFI bits for talking to database engines. Some will do it from scratch regardless, but I bet most database libraries would be happy to reuse the lowest level bits.

- better docs, as you've requested.

I hope to get started on this one.

- support for blobs/clobs

As Gregory just pointed out, using other libs adds further dependencies to Takusen. One of our earlier goals was to make it easy to install, which in the days before cabal meant fewer dependencies on external libs. cabal now mitigates that concern considerably, so perhaps we should relax more now when it comes to using external libs. Here is the list of things that I can think of right now which are currently internalised in Takusen, but which are also implementated in hackage libs: - extensible exceptions - EMonadIO - regions - iteratee

Refactoring the backends so that they can be added to the above lists as hackage packages would be very nice as well.

I'm not saying the way it is now is right or best, but it got there through various historical decisions, which were made with the best information available at the time, and in environments that differ from the current.

We should note these suggestions in the bugtracker so we don't lose sight of them: http://trac.haskell.org/takusen/report Thanks, Jason

On Sun, Aug 1, 2010 at 5:11 PM, Felipe Lessa wrote:

...

This same issues comes up fairly often on the darcs-users mailing list. My understanding of the way things are handled there, is that if there is ever a good reason to drop support for a version of GHC then the person who wants to drop support is supposed to look and see what is in Debian stable. If debian stable is still covered after the proposed changes then they are accepted automatically. If Debian stable would not be covered, then

On Sun, Aug 1, 2010 at 6:58 PM, Jason Dagit wrote: there

...

is a discussion to reach consensus. The reasons it works are: a) support is dropped lazily when there is a real demand to do so, instead of artificially; b) letting people discuss it case-by-case allows the decision to be made with full context.

Do you know if they test on GHC 6.8? Or do they just avoid extensions and hope it works? =)

They use a farm of buildbots to test: http://buildbot.darcs.net/ Jason