On Mon, Apr 26, 2004 at 08:43:44PM -0400, ajb@spamcop.net wrote:
And re sizeFM, I would note that Google has more than 2^31 pages indexed in memory.
I would note, in addition, that they don't have that many pages indexed on a single machine. Almost nobody has a database with that many records on a single machine, even those who have clusters of 64 bit machines.
It seems I qualify as almost nobody, cause I've had a database with more than 2^34 records on a single IA32 machine. However, I wouldn't ever think of loading the entire database to RAM. Best regards, Tom -- .signature: Too many levels of symbolic links
On Tue, 27 Apr 2004, Tomasz Zielonka wrote:
On Mon, Apr 26, 2004 at 08:43:44PM -0400, ajb@spamcop.net wrote:
I would note, in addition, that they don't have that many pages indexed on a single machine. Almost nobody has a database with that many records on a single machine, even those who have clusters of 64 bit machines.
It seems I qualify as almost nobody, cause I've had a database with more than 2^34 records on a single IA32 machine. However, I wouldn't ever think of loading the entire database to RAM.
Why not? Its only 17gb. If you are doing any substantive volume of transactions on that database* you may be better off storing the database in memory than buying the multiple CPUs and reliable RAID arrays you need to serve it off disk. A Dell PowerEdge 4600 w/ 24gb of memory and a write-caching controller w/ 6 36GB harddrives (RAID0 state,logging,status) costs around $34k. http://www1.us.dell.com/content/products/compare.aspx/tower?c=us&cs=555&l=en&s=biz Using e.g. a Prevayler style in memory logged data structures** you may be able to serve >100,000 queries/second. (see e.g. http://www.prevayler.org/wiki.jsp?topic=ScalabilityTestResults) To achieve just 500 queries/second using MaxDB (SAP-DB from MySQL) requires apparently 61!! App servers. (http://www.mysql.com/products/maxdb/benchmarks.html) Calculating the cost of this much hardware is left as an exercise for the reader, but, even ignoring the hardware costs here, you are also paying for rackspace and system administration to keep this many servers operating. -Alex- * OTOH, if this is just your personal mail archive then nevermind :- ) ** Finishing off a Haskell variant of this concept that also handles side-effects correctly right now. It is part II of the rdbms, appserver, web/mail server triad that I plan to use for a variety of different web apps. _________________________________________________________________ S. Alexander Jacobson mailto:me@alexjacobson.com tel:917-770-6565 http://alexjacobson.com
On Tue, Apr 27, 2004 at 02:05:06PM -0400, S. Alexander Jacobson wrote:
On Tue, 27 Apr 2004, Tomasz Zielonka wrote:
On Mon, Apr 26, 2004 at 08:43:44PM -0400, ajb@spamcop.net wrote:
It seems I qualify as almost nobody, cause I've had a database with more than 2^34 records on a single IA32 machine. However, I wouldn't ever think of loading the entire database to RAM.
Why not? Its only 17gb.
About 17000000000 records, yes, but not 17GB. It was more like 300 GB compressed data.
If you are doing any substantive volume of transactions on that database* you may be better off storing the database in memory than buying the multiple CPUs and reliable RAID arrays you need to serve it off disk.
I was not a typical relational database, so it's hard to compare. I managed on a 1 CPU P4 2.6 with 1GB mem and a RAID5 array. The database engine was optimised for bulk processing, reducing disk seeks and memory requirements.
A Dell PowerEdge 4600 w/ 24gb of memory and a write-caching controller w/ 6 36GB harddrives (RAID0 state,logging,status) costs around $34k. http://www1.us.dell.com/content/products/compare.aspx/tower?c=us&cs=555&l=en&s=biz Using e.g. a Prevayler style in memory logged data structures** you may be able to serve >100,000 queries/second. (see e.g. http://www.prevayler.org/wiki.jsp?topic=ScalabilityTestResults)
We were able to achieve a throughput of 200,000 records per seconds (not queries), which was about 3 times faster then necessary. The data you provided is interesting. However, if I had such a machine, my work could be much less challenging :)
* OTOH, if this is just your personal mail archive then nevermind :- )
No. Best regards, Tom -- .signature: Too many levels of symbolic links
On Tue, 27 Apr 2004, Tomasz Zielonka wrote:
The data you provided is interesting. However, if I had such a machine, my work could be much less challenging :)
I would note that people today are making strenuous efforts to retain compatibility with a Haskell98 standard that it only 6 years old! If we are making choices intended to have a lifetime of e.g. 10 years (see http://www.joelonsoftware.com/articles/fog0000000017.html), then I would observe that the 24gb machine I specced out will, assuming Moore's law lasts, cost only $2k in 5-6 years. The Sun Fire 15k with 288GB of memory and 72 Sparcs (think GPH!), that today costs $2.6m will cost only $155k. Note, since you don't need 72 CPUs, you'll probably be able to get this machine for <$50k. So, perhaps the question here is: What is the planned lifetime of sizeFM? Do we prefer the safety of knowing our code will work at arbitrary scale or the ecstacy of having it run fast today? Put differently, would you rather cope with hardware limitations today or software limitations tomorrow? -Alex- _________________________________________________________________ S. Alexander Jacobson mailto:me@alexjacobson.com tel:917-770-6565 http://alexjacobson.com
"S. Alexander Jacobson"
assuming Moore's law lasts
I suspect Moore's law is dead or dying. Intel just announced the latest "Prescott" P4 at 3.6 GHz, up from 3.2 last June. That's only a 12.5% increase. One year before that, it was about 2.5GHz, or almost 30%. I didn't recheck, but I think that in the PII and PIII period, the increase was about 50% every year. Interestingly, none of the other chipmakers seem to be doing much better either. But apart from that, your points are valid. :-) -kzm PS: Okay, that was perhaps off topic even for -cafe. -- If I haven't seen further, it is by standing in the footprints of giants
G'day all.
Quoting Tomasz Zielonka
It seems I qualify as almost nobody,
Yes, that's why I said "almost". I am also aware of a very, very small number of exceptions. (Disclaimer: I work in the database server area for a living, so I'm possibly only thinking of the needs of our customers, not necessarily other people.)
cause I've had a database with more than 2^34 records on a single IA32 machine.
I should qualify that slightly, and add the disclaimer that even if you are planning to put 2^34 records on a single box, it often pays to organise this as multiple physical databases with some kind of clustering layer on top, all on the one machine. Splitting the problem up increases the effectiveness of each CPU's L2 cache. Moreover, if your problem is easily partitioned (e.g. there are a lot of queries with a particularly highly discriminating key), any O(n log n) operations get effectively reduced to O(n/N log (n/N)).
However, I wouldn't ever think of loading the entire database to RAM.
For very large databases, smart software and smart operating systems mean that accessing RAM isn't that much less expensive than accessing disk these days because prediction is so good. The limiting factor is cache, and we won't see L2 cache size increasing anywhere near 17Gb any time soon. Cheers, Andrew Bromage
participants (5)
-
ajb＠spamcop.net -
Ketil Malde -
S. Alexander Jacobson -
S. Alexander Jacobson -
Tomasz Zielonka