On Thu, Aug 09, 2007 at 03:47:09PM -0400, Joe Uhl wrote: > We have a 30 GB database (according to pg_database_size) running nicely > on a single Dell PowerEdge 2850 right now. This represents data > specific to 1 US state. We are in the process of planning a deployment > that will service all 50 US states. > > If 30 GB is an accurate number per state that means the database size is > about to explode to 1.5 TB. About 1 TB of this amount would be OLAP > data that is heavy-read but only updated or inserted in batch. It is > also largely isolated to a single table partitioned on state. This > portion of the data will grow very slowly after the initial loading. > > The remaining 500 GB has frequent individual writes performed against > it. 500 GB is a high estimate and it will probably start out closer to > 100 GB and grow steadily up to and past 500 GB. What kind of transaction rate are you looking at? > I am trying to figure out an appropriate hardware configuration for such > a database. Currently I am considering the following: > > PowerEdge 1950 paired with a PowerVault MD1000 > 2 x Quad Core Xeon E5310 > 16 GB 667MHz RAM (4 x 4GB leaving room to expand if we need to) 16GB for 500GB of active data is probably a bit light. > PERC 5/E Raid Adapter > 2 x 146 GB SAS in Raid 1 for OS + logs. > A bunch of disks in the MD1000 configured in Raid 10 for Postgres data. > > The MD1000 holds 15 disks, so 14 disks + a hot spare is the max. With > 12 250GB SATA drives to cover the 1.5TB we would be able add another > 250GB of usable space for future growth before needing to get a bigger > set of disks. 500GB drives would leave alot more room and could allow > us to run the MD1000 in split mode and use its remaining disks for other > purposes in the mean time. I would greatly appreciate any feedback with > respect to drive count vs. drive size and SATA vs. SCSI/SAS. The price > difference makes SATA awfully appealing. Well, how does this compare with what you have right now? And do you expect your query rate to be 50x what it is now, or higher? > We plan to involve outside help in getting this database tuned and > configured, but want to get some hardware ballparks in order to get > quotes and potentially request a trial unit. You're doing a very wise thing by asking for information before purchasing (unfortunately, many people put that cart before the horse). This list is a great resource for information, but there's no real substitute for working directly with someone and being able to discuss your actual system in detail, so I'd suggest getting outside help involved before actually purchasing or even evaluating hardware. There's a lot to think about beyond just drives and memory with the kind of expansion you're looking at. For example, what ability do you have to scale past one machine? Do you have a way to control your growth rate? How well will the existing design scale out? (Often times what is a good design for a smaller set of data is sub-optimal for a large set of data.) Something else that might be worth looking at is having your existing workload modeled; that allows building a pretty accurate estimate of what kind of hardware would be required to hit a different workload. -- Decibel!, aka Jim Nasby decibel@xxxxxxxxxxx EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)
Attachment:
pgps4s5oqpX3p.pgp
Description: PGP signature
