Thank you for the answer! 1. acid-state, yes. 2. The application is supposed to work on the regular pc, without 32gb of ram. I agree that it doesn't cost much to buy 32gb of ram, but it's not the case. 3. Come to think of it, i think it would be more like about 10k elements per 10-20 second, yes. On average. On 05/06/2016 11:41 PM, Bardur Arantsson wrote:
On 05/06/2016 10:28 PM, Mikhail Volkhov wrote:
Hi!
I'm using ACID package as main database -- it's simple and... ACID (which is cool of course).
Just for clarification: Do you mean acid-state?
So now I need to store up to ~30GB of data in a single data structure (!) that will be constantly updated (some kind of a huge tree set). Here's a question -- how to operate that big structure? 1. It doesn't even fit in RAM
Yes it does. My desktop machine has 32GB RAM.
I'm not trying to brag or anything, and *you* may not have that amount of memory, but it *should* be cheap enough to buy enough RAM if it avoids radically redesigning an existing system (with all the associated risk, developer time, etc.).
If this is a one-off or rare occurrence, you could get 160GB RAM on an Amazon M4 instance (m4.10xlarge) if you wanted to. (Amazon's pricing structure is incredibly opaque and I didn't do a thorough investigation, so I apologize if this is wrong.)
2. It should be updated atomically and frequently (every 10 seconds up to 500 elements out of 10^7).
This is a bit vague. Do you mean that an *average* of 500 (out of 10^7) will be updated every 10 seconds?
3. What structures should I use? I'd like to store up to 10^6~10^7 some simple elements there too, that will be gigabytes of data. So it seems to me I can't use Data.Set.
Well, if storing things in RAM is out of the question, so is using Data.Set.
Regards,
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
-- Volkhov Mikhail M3#38 ITMO study group 17' Computer Technologies Department