Re: [Haskell-cafe] Re: HDBC or HSQL

...

I am asking because I am trying to make HAppS a reasonable replacement for all contexts in which you would otherwise use an external relational database except those in which an external SQL database is a specific requirement.

My experience with HAppS so far suggests that if one's data set is in the millions, an external database -- or some sort of off-ram storage mechanism is indeed a requirement, because even a 16GB server will eventually run out of memory. For example, I have a demo app (happstutorial.com) that stores jobs or users as records. On average, each record takes about 100B of memory, when checkpointed on hard disk. Not sure how much this amounts to in ram, but but on a 256M workstation with 256M of virtual memory, the checkpointing mechanism had an out of memory error with under 500,000 records. This doesn't seem totally unreasonable to me: 500,000 * 100 = 50,000,000B, which is 20% of my physical memory and 10% of my total (including virtual) memory. Also, as my recent post to haskell cafe says, I can't even fit a 20M element Data.Map.Map Int Int object into memory on my 256M laptop. On a 16GB server, you might get 32X the limit I ran into, but you will still have out of memory errors with > 15 million records. Perhaps a recordset in the millions isn't a "reasonable" requirement, but most web 2.0 type projects are of this scope, I think. Thomas. 2007/8/4 Alex Jacobson :

Have you looked at the HAppS.DBMS.IxSet? It gives you a type safe way to query indexed collections.

-Alex-

Isto Aho wrote:

...

Hi,

I'd like to store small matrices into a db. Number of rows and columns may vary in a way not known in advance. One might use a relation (matrixId, col, row, value) or something like that but if it is possible to put a matrix in one command into db, some queries will be easier. E.g., one relation can store several matrices and it would be easy to query, how many matrices are stored currently. With that above four tuple you can find out the number of unique matrixId's, too, but it is not as easy as with matrices.

Anyhow, now I'm not sure if I should stick with HSQL any more... Earlier comments on this thread made me think that maybe it would be a better idea to try to learn enough HDBC.

This would be used in a server application. Is HAppS applicable here?

e.g. after some tweaking the following works with HSQL:

addRows = do dbh <- connect server database user_id passwd intoDB dbh ([555,111, 50, 1000]::[Int]) ([21.0,22.0,23.0,24.0]::[Double]) intoDB dbh ([556,111, 50, 1000]::[Int]) ([21.0,22.0,23.0,24.0]::[Double]) intoDB dbh ([]::[Int]) ([]::[Double]) where intoDB dbh i_lst d_lst = catchSql (do let cmd = "INSERT INTO trial (intList, dList) VALUES (" ++ toSqlValue i_lst ++ "," ++ toSqlValue d_lst ++ ")" execute dbh cmd ) (\e -> putStrLn $ "Problem: " ++ show e)

Similarly, queries can handle matrices and I like that it is now possible to select those columns or rows from the stored matrix that are needed. E.g.

retrieveRecords2 :: Connection -> IO [[Double]] retrieveRecords2 c = do -- query c "select dList[1:2] from trial" >>= collectRows getRow query c "select dList from trial" >>= collectRows getRow where getRow :: Statement -> IO [Double] getRow stmt = do lst <- getFieldValue stmt "dList" return lst readTable2 = do dbh <- connect server database user_id passwd values <- retrieveRecords2 dbh putStrLn $ "dLists are : " ++ (show values)

br, Isto

2007/8/1, Alex Jacobson mailto:alex@alexjacobson.com>:

Out of curiosity, can I ask what you are actually trying to do?

I am asking because I am trying to make HAppS a reasonable replacement for all contexts in which you would otherwise use an external relational database except those in which an external SQL database is a specific requirement.

-Alex-

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe