On 08/23/12 11:24 AM, Sébastien Lorion wrote:
I think both kind of tests (general and app specific) are complementary and useful in their own way. At a minimum, if the general ones fail, why go to the expenses of doing the specific ones ? Setting up a meaningful application test can take a lot of time and it can be hard to pinpoint exactly where in the stack the performance drops occur. The way I see it, synthetic benchmarks allow to isolate somewhat the layers and serve as a base to validate application tests done later on. It surprises me that asking for the general perf behavior of a platform is controversial.
I don't use AWS at all. But, it shouldnt take more than a couple hours to spin up an instance, populate a pgbench database and run a series of pgbench runs against it, and do the same against any other sort of system you wish to use as your reference.
I like to test with a database about twice the size of the available memory if I'm testing IO, and I've found that pgbench -i -s ####, for ####=10000 it generates a 1 billion row table and uses about 150GB (and a hour or so to initialize on fast IO hardware). I then run pgbench with -c of about 2-4X the cpu/thread count, and -j of about -c/16, and a -t of at least 10000 (so each client connection runs 10000 transactions).
on a modest but decent 2U class 2-socket dedicated server with a decent raid card and raid10 across enough spindles, I can see numbers as high as 5000 transactions/second with 15krpm rust, and 7000-8000 with a couple MLC SSD's striped. trying to raid10 a bunch of SATA 7200 disks gives numbers more like 1000. using host based raid, without a write-back cache in the raid card, gives numbers about 1/2 the above. the IOPS during these tests hit around 12000 or 15000 small writes/second.
doing this level of IO on a midsized SAN can often cause the SAN CPU to run at 80%+ so if there's other activity on the SAN from other hosts, good luck.
in a heavily virtualized shared-everything environment, I'm guessing your numbers will be all over the place and difficult to achieve consistency.
-- john r pierce N 37, W 122 santa cruz ca mid-left coast -- Sent via pgsql-general mailing list (pgsql-general@xxxxxxxxxxxxxx) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
