Mike Bresnahan wrote:
I have deployed PostgresSQL 8.4.1 on a Fedora 9 c1.xlarge (8x1 cores) instance in the Amazon E2 Cloud. When I run pgbench in read-only mode (-S) on a small database, I am unable to peg the CPUs no matter how many clients I throw at it. In fact, the CPU utilization never drops below 60% idle. I also tried this on Fedora 12 (kernel 2.6.31) and got the same basic result. What's going on here? Am I really only utilizing 40% of the CPUs? Is this to be expected on virtual (xen) instances? tps = 19663.841772 (including connections establishing
Looks to me like you're running into a general memory bandwidth issue here, possibly one that's made a bit worse by how pgbench works. It's a somewhat funky workload Linux systems aren't always happy with, although one of your tests had the right configuration to sidestep the worst of the problems there. I don't see any evidence that pgbench itself is a likely suspect for the issue, but it does shuffle a lot of things around in memory relative to transaction time when running this small select-only test, and clients can get stuck waiting for it when that happens.
To put your results in perspective, I would expect to get around 25K TPS running the pgbench setup/test you're doing on a recent 4-core/single processor system, and around 50K TPS is normal for an 8-core server doing this type of test. And those numbers are extremely sensitive to the speed of the underlying RAM even with the CPU staying the same.
I would characterize your results as "getting about 1/2 of the CPU+memory performance of an install on a dedicated 8-core system". That's not horrible, as long as you have reasonable expectations here, which is really the case for any virtualized install I think. I'd actually like to launch a more thorough investigation into this particular area, exactly how the PostgreSQL bottlenecks shift around on EC2 compared to similar dedicated hardware, if I found a sponsor for it one day. A bit too much work to do it right just for fun.
-- Greg Smith 2ndQuadrant Baltimore, MD PostgreSQL Training, Services and Support greg@xxxxxxxxxxxxxxx www.2ndQuadrant.com -- Sent via pgsql-general mailing list (pgsql-general@xxxxxxxxxxxxxx) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
