Hi all, I have some program data that I'd like to persist. I could just use read and show and file I/O to store arrays as files. [1] This works and it is easy and simple (which I like) but it is also inefficient and a little cumbersome. I imagine that this is a very common task, and I know of utilities in other environments like object databases, embedded databases, XML persistence, etc. that can make this kind of persistence simple *and* relatively efficient and reliable. E.g.) I have found the HSQLDB embedded database [2] to be useful when writing Java applications. What do you do for quick and easy persistence in Haskell? Are there any libraries or techniques that are especially useful for this? [1]
type SimpleRecord = (Integer, Integer, Integer) type SimpleIndex = Integer type SimpleDB = (Array SimpleIndex SimpleRecord, FilePath)
loadDB :: FilePath -> IO SimpleDB loadDB f = do h <- openFile f ReadMode s <- hGetContents h hClose h return $ r s where r :: String -> SimpleDB r = read
getRecord :: SimpleDB -> SimpleIndex -> SimpleRecord getRecord (a, _) = (a !)
updateRecord :: SimpleDB -> SimpleIndex -> SimpleRecord -> SimpleDB updateRecord (a, f) i r = (a//[(i,r)],f)
storeDB :: SimpleDB -> IO () storeDB sdb@(a,f) = do h <- openFile f WriteMode hPutStr h (show sdb) hClose h
sampleDB = (array (0,10) [(x,(x+1,x*2,42)) | x <- [0..10]],"output/ 1.db.txt")
[2] http://hsqldb.org/ - Matt Munz mmm92@pantheon.yale.edu
I usually use the the Binary class, found in NewBinary for this task. You derive Binary for each type you wish to serialise, which gives you a get and put function on handles. A stripped down version suitable for many tasks lives here: http://www.cse.unsw.edu.au/~dons/code/hmp3/Binary.hs This is the same class GHC uses to read and write .hi files. Some info on this at http://www.haskell.org/hawiki/BinaryIo, and ftp://ftp.cs.york.ac.uk/pub/malcolm/ismm98.html Examples of a persistent database here: http://www.cse.unsw.edu.au/~dons/code/hmp3/Tree.hs and here http://www.cse.unsw.edu.au/~dons/code/lambdabot/Plugins/Seen.hs I actually think a derivable class Binary is simple and solid enough it should be in the standard libs.. -- Don mmm92:
Hi all,
I have some program data that I'd like to persist. I could just use read and show and file I/O to store arrays as files. [1] This works and it is easy and simple (which I like) but it is also inefficient and a little cumbersome.
I imagine that this is a very common task, and I know of utilities in other environments like object databases, embedded databases, XML persistence, etc. that can make this kind of persistence simple *and* relatively efficient and reliable. E.g.) I have found the HSQLDB embedded database [2] to be useful when writing Java applications. What do you do for quick and easy persistence in Haskell? Are there any libraries or techniques that are especially useful for this?
[1]
type SimpleRecord = (Integer, Integer, Integer) type SimpleIndex = Integer type SimpleDB = (Array SimpleIndex SimpleRecord, FilePath)
loadDB :: FilePath -> IO SimpleDB loadDB f = do h <- openFile f ReadMode s <- hGetContents h hClose h return $ r s where r :: String -> SimpleDB r = read
getRecord :: SimpleDB -> SimpleIndex -> SimpleRecord getRecord (a, _) = (a !)
updateRecord :: SimpleDB -> SimpleIndex -> SimpleRecord -> SimpleDB updateRecord (a, f) i r = (a//[(i,r)],f)
storeDB :: SimpleDB -> IO () storeDB sdb@(a,f) = do h <- openFile f WriteMode hPutStr h (show sdb) hClose h
sampleDB = (array (0,10) [(x,(x+1,x*2,42)) | x <- [0..10]],"output/ 1.db.txt")
- Matt Munz mmm92@pantheon.yale.edu
_______________________________________________ Haskell mailing list Haskell@haskell.org http://www.haskell.org/mailman/listinfo/haskell
"Matthew M. Munz" <mmm92@pantheon.yale.edu> writes:
I have some program data that I'd like to persist. I could just use read and show and file I/O to store arrays as files. [1] This works and it is easy and simple (which I like) but it is also inefficient and a little cumbersome.
I like SerTH - http://www.cs.helsinki.fi/u/ekarttun/SerTH/ -- Shae Matijs Erisson - http://www.ScannedInAvian.com/ - Sockmonster once said: You could switch out the unicycles for badgers, and the game would be the same.
Hello Matthew, Wednesday, December 07, 2005, 5:28:42 AM, you wrote: MMM> I have some program data that I'd like to persist. I could just there is 3 binary serialization packages for GHC: 1) GhcBinary, which is most widely used (including GHC compiler itself) and is close to be "standard de-facto" 2) SerTH (http://www.cs.helsinki.fi/u/ekarttun/SerTH/SerTH-0.2.tar.gz), which unique seling points are serialization of cyclic datastructures and automatic derivation of serialization code for your data types using Template Haskell 3) my own ByteStream (http://freearc.narod.ru), which is oriented toward fast working and using non-trivial i/o methods (in my own program serialized data are sent to another thread which compress them with gzip-like algorithm) is it what you asked for? -- Best regards, Bulat mailto:bulatz@HotPOP.com
Donald, Shae, and Bulat, It sounds like Binary is the right tool for this job. Thanks for all of your helpful suggestions. - Matt On Dec 7, 2005, at 8:36 AM, Bulat Ziganshin wrote:
Hello Matthew,
Wednesday, December 07, 2005, 5:28:42 AM, you wrote:
MMM> I have some program data that I'd like to persist. I could just
there is 3 binary serialization packages for GHC:
1) GhcBinary, which is most widely used (including GHC compiler itself) and is close to be "standard de-facto"
2) SerTH (http://www.cs.helsinki.fi/u/ekarttun/SerTH/ SerTH-0.2.tar.gz), which unique seling points are serialization of cyclic datastructures and automatic derivation of serialization code for your data types using Template Haskell
3) my own ByteStream (http://freearc.narod.ru), which is oriented toward fast working and using non-trivial i/o methods (in my own program serialized data are sent to another thread which compress them with gzip-like algorithm)
is it what you asked for?
-- Best regards, Bulat mailto:bulatz@HotPOP.com
Hello Guys, I am looking at SerTH. The serialization of cyclic datastructures is quite useful feature but since I did some performance testing I found that it is too slow and consumes a lot of memory. It isn't surprising, since the cycles detection has extra overhead. I have changed the SerTH sources and now the cycles detection is optional. Unfortunatelly it is still about two times slower than NewBinary. In the same time the NewBinary has the advantage that it can serialize separate bits to the stream. What I want is to have a single Binary library that is fast and that have all these features: - fast implementation - optional serialization of cyclic datastructures - optional support for bit level serialization - automatic class derivation based on TH If there are voluntiers I would like to cooperate in the development. Cheers, Krasimir 2005/12/8, Matthew M. Munz <mmm92@pantheon.yale.edu>:
Donald, Shae, and Bulat,
It sounds like Binary is the right tool for this job. Thanks for all of your helpful suggestions.
- Matt
On Dec 7, 2005, at 8:36 AM, Bulat Ziganshin wrote:
Hello Matthew,
Wednesday, December 07, 2005, 5:28:42 AM, you wrote:
MMM> I have some program data that I'd like to persist. I could just
there is 3 binary serialization packages for GHC:
1) GhcBinary, which is most widely used (including GHC compiler itself) and is close to be "standard de-facto"
2) SerTH (http://www.cs.helsinki.fi/u/ekarttun/SerTH/ SerTH-0.2.tar.gz), which unique seling points are serialization of cyclic datastructures and automatic derivation of serialization code for your data types using Template Haskell
3) my own ByteStream (http://freearc.narod.ru), which is oriented toward fast working and using non-trivial i/o methods (in my own program serialized data are sent to another thread which compress them with gzip-like algorithm)
is it what you asked for?
-- Best regards, Bulat mailto:bulatz@HotPOP.com
_______________________________________________ Haskell mailing list Haskell@haskell.org http://www.haskell.org/mailman/listinfo/haskell
Hello Krasimir, Thursday, December 08, 2005, 5:40:15 PM, you wrote: KA> the stream. What I want is to have a single Binary library that is KA> fast and that have all these features: KA> - fast implementation KA> - optional serialization of cyclic datastructures KA> - optional support for bit level serialization KA> - automatic class derivation based on TH KA> If there are voluntiers I would like to cooperate in the development. imho, it's better to start from NewBinary library and add features of SerTH to it. i can make TH support. btw, where i can download NewBinary version about you talk? there is many different variations. one disadvantage of original Binary library as well as all its derivatives is support only for memory- and Handle-oriented streams. my library instead accepts any stream, you just provide functions to read/write when create Binary object. i propose to add this facility too, smth like: createBinary readF writeF = return Binary {readF=readF,writeF=writeF} createBinary4File h = createBinary (hGetChar h) (hPutChar h) createBinary4Memory addr size = do ptr <- newIORef addr let read = do ... write = do ... createBinary read write -- Best regards, Bulat mailto:bulatz@HotPOP.com
2005/12/8, Bulat Ziganshin <bulatz@hotpop.com>:
imho, it's better to start from NewBinary library and add features of SerTH to it. i can make TH support. btw, where i can download NewBinary version about you talk? there is many different variations.
Thanks. Your help will be appreciable. I am talking about JeremyShaw's update of ?HalDaume's library: http://www.n-heptane.com/nhlab/repos/NewBinary/
one disadvantage of original Binary library as well as all its derivatives is support only for memory- and Handle-oriented streams. my library instead accepts any stream, you just provide functions to read/write when create Binary object. i propose to add this facility too, smth like:
createBinary readF writeF = return Binary {readF=readF,writeF=writeF}
I think the right abstraction should use carefully designed type classes like in Simon Marlow's NewIO library. Cheer
At Thu, 8 Dec 2005 16:40:15 +0200, Krasimir Angelov wrote:
Hello Guys,
If there are voluntiers I would like to cooperate in the development.
Hello, I would be interested in helping with this a bit. Conveniently, I currently maintain NewBinary :p My first order of business will be to update the library and cabal stuff a bit[1], and figure out how to automatically generate tarballs and changelogs for the website. If anyone has any quick requests for updates, let me know, I plan to do my updates this Sunday. Jeremy Shaw. [1] I plan to switch Setup.lhs to use runhaskell instead of runhugs. And I am going to look into getting rid of the IOExt deprecated warnings.
participants (6)
-
Bulat Ziganshin -
dons＠cse.unsw.edu.au -
Jeremy Shaw -
Krasimir Angelov -
Matthew M. Munz -
Shae Matijs Erisson