On Wed, Jan 27, 2010 at 3:59 PM, Mike Bresnahan <mike.bresnahan@xxxxxxxxxxx> wrote:
I have deployed PostgresSQL 8.4.1 on a Fedora 9 c1.xlarge (8x1 cores) instance
in the Amazon E2 Cloud. When I run pgbench in read-only mode (-S) on a small
database, I am unable to peg the CPUs no matter how many clients I throw at it.
In fact, the CPU utilization never drops below 60% idle. I also tried this on
Fedora 12 (kernel 2.6.31) and got the same basic result. What's going on here?
Am I really only utilizing 40% of the CPUs? Is this to be expected on virtual
(xen) instances?
I have seen behavior like this in the past on EC2. I believe your bottleneck may be pulling the data out of cache. I benchmarked this a while back and found that memory speeds are not much faster than disk speeds on EC2. I am not sure if that is true of Xen in general or if its just limited to the cloud.
[root@domU-12-31-39-0C-88-C1 ~]# uname -a
Linux domU-12-31-39-0C-88-C1 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20
17:48:28 EST 2009 x86_64 x86_64 x86_64 GNU/Linux
-bash-4.0# pgbench -S -c 16 -T 30 -h domU-12-31-39-0C-88-C1 -U postgres
Password:
starting vacuum...end.
transaction type: SELECT only
scaling factor: 64
query mode: simple
number of clients: 16
duration: 30 s
number of transactions actually processed: 590508
tps = 19663.841772 (including connections establishing)
tps = 19710.041020 (excluding connections establishing)
top - 15:55:05 up 1:33, 2 users, load average: 2.44, 0.98, 0.44
Tasks: 123 total, 11 running, 112 sleeping, 0 stopped, 0 zombie
Cpu(s): 18.9%us, 8.8%sy, 0.0%ni, 70.6%id, 0.0%wa, 0.0%hi, 1.7%si, 0.0%st
Mem: 7348132k total, 1886912k used, 5461220k free, 34432k buffers
Swap: 0k total, 0k used, 0k free, 1456472k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2834 postgres 15 0 191m 72m 70m S 16 1.0 0:00.66 postmaster
2838 postgres 15 0 191m 66m 64m R 15 0.9 0:00.62 postmaster
2847 postgres 15 0 191m 70m 68m S 15 1.0 0:00.59 postmaster
2837 postgres 15 0 191m 72m 70m S 14 1.0 0:00.47 postmaster
2842 postgres 15 0 191m 66m 64m R 14 0.9 0:00.48 postmaster
2835 postgres 15 0 191m 69m 67m S 14 1.0 0:00.54 postmaster
2839 postgres 15 0 191m 69m 67m R 14 1.0 0:00.60 postmaster
2840 postgres 15 0 191m 68m 67m R 14 1.0 0:00.58 postmaster
2833 postgres 15 0 191m 68m 66m R 14 1.0 0:00.50 postmaster
2845 postgres 15 0 191m 70m 68m R 14 1.0 0:00.50 postmaster
2846 postgres 15 0 191m 67m 65m R 14 0.9 0:00.51 postmaster
2836 postgres 15 0 191m 66m 64m S 12 0.9 0:00.43 postmaster
2844 postgres 15 0 191m 68m 66m R 11 1.0 0:00.40 postmaster
2841 postgres 15 0 191m 65m 64m R 11 0.9 0:00.43 postmaster
2832 postgres 15 0 191m 67m 65m S 10 0.9 0:00.38 postmaster
2843 postgres 15 0 191m 67m 66m S 10 0.9 0:00.43 postmaster
[root@domU-12-31-39-0C-88-C1 ~]# iostat -d 2 -x
Linux 2.6.21.7-2.ec2.v1.2.fc8xen (domU-12-31-39-0C-88-C1) 01/27/10
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz
avgqu-sz await svctm %util
sda1 0.57 15.01 1.32 3.56 34.39 148.57 37.52
0.28 57.35 3.05 1.49
sdb1 0.03 112.38 5.50 12.11 87.98 995.91 61.57
1.88 106.61 2.23 3.93
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz
avgqu-sz await svctm %util
sda1 0.00 0.00 0.00 1.79 0.00 28.57 16.00
0.00 2.00 1.50 0.27
sdb1 0.00 4.46 0.00 14.29 0.00 150.00 10.50
0.37 26.00 2.56 3.66
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz
avgqu-sz await svctm %util
sda1 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00
sdb1 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz
avgqu-sz await svctm %util
sda1 0.00 3.57 0.00 0.79 0.00 34.92 44.00
0.00 3.00 3.00 0.24
sdb1 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00
--
Sent via pgsql-general mailing list (pgsql-general@xxxxxxxxxxxxxx)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general
--
--
Jim Mlodgenski
EnterpriseDB (http://www.enterprisedb.com)
