gue.schmidt:
Hi,
well I tried to do some stuff in memory, and the app ended up using a couple of gigs. I not only have a very large amount of dynamic data, CSV files, but also quite a large amount of static data, and wasted 3 months trying to do this all in-memory. The problem was finally solved once I used SQLite and SQL.
The other day I had one last go at trying to compile the static data in a literal list in my haskell code. That was 80.000 rows, it was just not even possible
Don't compile in static data (or if you do, use -Onot, so that GHC won't try to analyze it)! Use some kind of binary on-disk storage.
As far as I'm concerned this discussion is settled in favor of SQL once and for all.
The part I didn't like about SQLite is encryption, you need to buy that extra and then hope that it fits the current version and future ones too. HSQLDB or Derby for Java give you this option and also with in-memory database, alas they are for Java only.
You might also want to look at the HAppS disk-backed persistence model, http://hackage.haskell.org/package/HAppS-State Or the holumbus distributed storage layer, http://hackage.haskell.org/package/Holumbus-Storage