-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi list,
please forgive cross posting, but I cannot specify the problem enough to
say whether list it fits perfectly, so I'll ask on both.
I have some machines based with following specs (see at the end of the
email).
They run CentOS 5.4 x86_64 with the latest patches applied, Xen-enabled
and should host one or more domUs. I put the domUs' storage on LVM, as I
learnt ages ago (what never caused any problems) and is way faster than
using file-based 'images'.
However, there's something special about these machines: They have the
new WD EARS series drives, which use 4K sector sizes. So, I booted a
rescue system and used fdisk to start at sector 64 instead of 63 (long
story made short: Due to overhead causing the drive to do much more,
inefficient writes when starting at sector 63, the performance
collapses; with 'normal' geometry (sector 63), the drive achieves about
25MiByte/sec writes, with starting at sector 64 partition, it achieves
almost 100MiByte/sec writes):
[root@server2 ~]# fdisk -ul /dev/sda
Disk /dev/sda: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders, total 1953525168 sectors
Units = sectors of 1 * 512 = 512 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 64 2097223 1048580 fd Linux raid
autodetect
Partition 1 does not end on cylinder boundary.
/dev/sda2 2097224 18876487 8389632 82 Linux swap / Solaris
/dev/sda3 18876488 1953525167 967324340 fd Linux raid
autodetect
On top of those (two per machine) WD EARS HDs there's ``md'' providing
two RAID1, /boot and LVM, as well as swap per HD (i.e. non-RAIDed). LVM
provides the / partition as well as LVs for Xen domUs.
I have about 60 machines running that style and never had any problems.
They run like a charm. On these machines, however, domUs are *very*
slow, have a steady (!) load of about two -- 50% stating in 'wait' --
and all operations take ages, e.g. a ``yum update'' with the recently
released updates.
Now, can that be due to 4K issues I didn't see, nestet now in LVM?
Help is very appreciated.
Cheers,
Timo
- ---
Linux server2.blah.org 2.6.18-164.11.1.el5xen #1 SMP Wed Jan 20 08:06:04
EST 2010 x86_64 x86_64 x86_64 GNU/Linux
- ---
[root@server2 ~]# cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel(R) Core(TM)2 Quad CPU Q9400 @ 2.66GHz
stepping : 10
cpu MHz : 1998.000
cache size : 3072 KB
physical id : 0
siblings : 1
core id : 0
cpu cores : 1
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu tsc msr pae mce cx8 apic mtrr mca cmov pat pse36
clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc
pni monitor ds_cpl vmx smx est tm2 cx16 xtpr lahf_lm
bogomips : 6668.58
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel(R) Core(TM)2 Quad CPU Q9400 @ 2.66GHz
stepping : 10
cpu MHz : 1998.000
cache size : 3072 KB
physical id : 1
siblings : 1
core id : 0
cpu cores : 1
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu tsc msr pae mce cx8 apic mtrr mca cmov pat pse36
clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc
pni monitor ds_cpl vmx smx est tm2 cx16 xtpr lahf_lm
bogomips : 6668.58
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
processor : 2
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel(R) Core(TM)2 Quad CPU Q9400 @ 2.66GHz
stepping : 10
cpu MHz : 1998.000
cache size : 3072 KB
physical id : 2
siblings : 1
core id : 0
cpu cores : 1
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu tsc msr pae mce cx8 apic mtrr mca cmov pat pse36
clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc
pni monitor ds_cpl vmx smx est tm2 cx16 xtpr lahf_lm
bogomips : 6668.58
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
processor : 3
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel(R) Core(TM)2 Quad CPU Q9400 @ 2.66GHz
stepping : 10
cpu MHz : 1998.000
cache size : 3072 KB
physical id : 3
siblings : 1
core id : 0
cpu cores : 1
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu tsc msr pae mce cx8 apic mtrr mca cmov pat pse36
clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc
pni monitor ds_cpl vmx smx est tm2 cx16 xtpr lahf_lm
bogomips : 6668.58
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
- ---
[root@server2 ~]# cat /proc/meminfo
MemTotal: 524288 kB
MemFree: 80620 kB
Buffers: 23352 kB
Cached: 205400 kB
SwapCached: 0 kB
Active: 132448 kB
Inactive: 156424 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 524288 kB
LowFree: 80620 kB
SwapTotal: 16779248 kB
SwapFree: 16779248 kB
Dirty: 32 kB
Writeback: 0 kB
AnonPages: 60112 kB
Mapped: 13348 kB
Slab: 30996 kB
PageTables: 4424 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
CommitLimit: 17041392 kB
Committed_AS: 334800 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 12572 kB
VmallocChunk: 34359724607 kB
- ---
[root@server2 ~]# fgrep min-mem /etc/xen/xend-config.sxp
# dom0-min-mem is the lowest memory level (in MB) dom0 will get down to.
# If dom0-min-mem=0, dom0 will never balloon out.
(dom0-min-mem 512)
- ---
[root@server2 ~]# fgrep dom0 /boot/grub/menu.lst
kernel /xen.gz-2.6.18-164.11.1.el5 dom0_mem=512M
- ---
example of ``dstat'' running while ``yum update'' was done; I think the
CPU is in ``wait'' state too much:
- ----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read writ| recv send| in out | int csw
0 0 100 0 0 0| 0 0 | 136B 178B| 0 0 | 11 12
0 0 100 0 0 0| 0 0 | 966B 892B| 0 0 | 35 31
12 5 83 0 0 0| 0 0 |4068B 4521B| 0 0 | 245 150
47 4 50 0 0 0| 0 0 | 126B 178B| 0 0 | 113 11
46 3 51 0 0 0| 0 0 | 328B 470B| 0 0 | 133 22
0 0 73 28 0 0| 48k 0 | 198B 454B| 0 0 | 30 27
41 3 51 5 0 0| 192k 0 | 522B 1246B| 0 0 | 164 61
9 2 89 0 0 0|8192B 968k| 630B 1896B| 0 0 | 62 35
0 0 100 0 0 0| 0 0 | 136B 178B| 0 0 | 15 16
0 0 100 0 0 0| 0 0 | 246B 292B| 0 0 | 14 17
1 0 99 0 0 0| 0 0 |1231k 28k| 0 0 |1004 925
0 0 100 0 0 0| 0 0 |3394k 77k| 0 0 |2871 2943
27 5 48 20 0 0| 968k 0 | 442k 10k| 0 0 | 641 657
19 5 59 17 0 0| 344k 536k|1644B 4064B| 0 0 | 414 339
0 0 50 50 0 0| 56k 320k| 186B 232B| 0 0 | 128 129
0 1 44 54 0 0| 136k 1312k| 278B 220B| 0 0 | 126 107
0 0 55 45 0 0|1552k 11M| 126B 178B| 0 0 | 502 139
0 0 53 48 0 0| 568k 0 | 126B 178B| 0 0 | 41 32
0 0 50 50 0 0| 0 0 | 126B 178B| 0 0 | 16 14
1 1 53 46 0 0|9608k 0 | 258B 566B| 0 0 |1473 2456
12 3 54 32 0 0|1368k 0 |2112B 6064B| 0 0 | 713 603
12 1 36 52 0 0| 888k 1192k| 858B 2426B| 0 0 | 394 429
0 0 52 48 0 0| 0 2472k| 126B 178B| 0 0 | 189 75
- ----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read writ| recv send| in out | int csw
0 0 54 46 0 0| 0 728k| 66B 178B| 0 0 | 107 30
0 0 41 59 0 0|8192B 448k| 126B 322B| 0 0 | 85 46
0 0 55 45 0 0| 0 1920k| 126B 178B| 0 0 | 185 72
0 0 54 46 0 0| 0 2688k| 126B 178B| 0 0 | 238 78
0 0 41 59 0 0| 0 1576k| 126B 178B| 0 0 | 136 51
0 0 47 53 0 0|8192B 2128k| 66B 178B| 0 0 | 207 53
0 0 50 50 0 0| 40k 2744k| 66B 178B| 0 0 | 277 60
0 0 50 50 0 0| 16k 3536k| 66B 178B| 0 0 | 330 59
0 0 50 50 0 0|8192B 1016k| 66B 178B| 0 0 | 98 16
0 0 59 41 0 0| 80k 4320k| 66B 178B| 0 0 | 108 100
0 0 48 52 0 0| 16k 208k| 126B 178B| 0 0 | 80 89
0 0 46 54 0 0| 56k 0 | 308B 178B| 0 0 | 38 68
0 0 42 58 0 0| 0 0 | 66B 178B| 0 0 | 11 11
0 0 54 45 0 0|1264k 752k| 66B 178B| 0 0 | 282 428
0 0 53 47 0 0| 0 360k| 66B 178B| 0 0 | 81 101
0 0 53 46 0 0| 0 576k| 66B 178B| 0 0 | 141 129
0 0 38 62 0 0| 536k 88k| 126B 178B| 0 0 | 68 62
0 0 50 50 0 0| 0 0 | 66B 178B| 0 0 | 17 15
1 1 52 46 0 0| 336k 392k| 126B 178B| 0 0 | 105 115
0 0 39 61 0 0| 152k 3504k| 126B 178B| 0 0 | 199 63
0 0 49 51 0 0| 40k 992k| 186B 178B| 0 0 | 122 40
0 0 56 44 0 0| 0 216k| 186B 178B| 0 0 | 73 39
0 0 42 58 0 0| 0 224k| 66B 178B| 0 0 | 69 30
- ----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read writ| recv send| in out | int csw
0 0 50 50 0 0| 0 216k| 66B 178B| 0 0 | 89 36
0 0 51 50 0 0| 0 272k| 126B 322B| 0 0 | 100 34
0 0 50 50 0 0|8192B 312k| 66B 178B| 0 0 | 81 36
0 0 56 44 0 0| 0 560k| 186B 178B| 0 0 | 103 44
0 0 43 57 0 0| 0 488k| 126B 178B| 0 0 | 91 16
0 0 50 50 0 0| 0 408k| 126B 178B| 0 0 | 59 12
0 0 64 36 0 0| 72k 120k| 380B 566B| 0 0 | 140 87
0 0 44 56 0 0| 0 0 | 66B 178B| 0 0 | 9 10
0 0 55 45 0 0| 16k 0 | 126B 178B| 0 0 | 32 38
2 0 21 78 0 0| 72k 744k| 846B 2038B| 0 0 | 243 275
0 0 44 56 0 0| 0 0 | 186B 178B| 0 0 | 15 19
0 0 50 50 0 0|8192B 992k| 66B 178B| 0 0 | 72 16
0 1 51 48 0 0| 440k 568k| 126B 178B| 0 0 | 105 142
0 0 65 34 0 0| 32k 48k| 192B 372B| 0 0 | 41 43
2 1 41 56 0 0| 80k 872k|1254B 3784B| 0 0 | 271 264
0 0 44 56 0 0| 0 264k| 126B 178B| 0 0 | 66 84
0 0 61 39 0 0| 64k 224k| 126B 178B| 0 0 | 125 162
1 1 18 81 0 0| 0 736k| 132B 372B| 0 0 | 88 35
0 0 59 41 0 0| 104k 912k|1032B 2312B| 0 0 | 176 113
0 0 44 56 0 0| 0 0 |1090B 178B| 0 0 | 15 13
1 0 57 41 0 0| 96k 704k| 528B 1456B| 0 0 | 167 206
0 0 44 56 0 0| 40k 16k|1270B 178B| 0 0 | 39 36
0 0 50 50 0 0| 0 0 | 126B 178B| 0 0 | 13 13
- ----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read writ| recv send| in out | int csw
0 0 50 50 0 0| 16k 2368k| 66B 178B| 0 0 | 165 50
1 0 54 45 0 0| 72k 304k|1692B 4510B| 0 0 | 248 201
0 0 54 46 0 0|8192B 176k|3898B 178B| 0 0 | 132 83
0 0 51 49 0 0|8192B 240k|7236B 252B| 0 0 | 212 101
0 0 50 50 0 0| 40k 0 | 126B 178B| 0 0 | 27 31
0 0 32 68 0 0| 0 0 |3034B 178B| 0 0 | 61 15
0 3 63 34 0 0| 280k 1840k|6350B 820B| 0 0 | 378 354
0 0 44 56 0 0| 0 336k| 66B 178B| 0 0 | 73 88
0 0 50 50 0 0|8192B 248k| 66B 178B| 0 0 | 62 52
0 0 50 50 0 0| 336k 200k| 126B 178B| 0 0 | 65 71
0 0 55 45 0 0| 72k 368k| 126B 178B| 0 0 | 80 100
0 0 52 48 0 0| 192k 176k| 66B 178B| 0 0 | 54 69
0 0 41 59 0 0| 112k 272k| 66B 178B| 0 0 | 71 64
0 0 40 60 0 0| 0 0 | 126B 178B| 0 0 | 25 24
0 0 57 43 0 0| 240k 216k| 186B 330B| 0 0 | 63 86
0 0 51 49 0 0| 120k 808k| 126B 178B| 0 0 | 131 157
0 0 50 50 0 0| 0 296k| 66B 178B| 0 0 | 65 84
0 0 50 50 0 0| 0 296k| 126B 178B| 0 0 | 72 98
0 0 41 59 0 0| 0 2848k| 126B 178B| 0 0 | 154 112
0 0 50 50 0 0| 0 384k| 188B 178B| 0 0 | 84 32
0 0 53 47 0 0| 0 272k|1812B 178B| 0 0 | 82 17
0 0 47 53 0 0| 0 208k| 196B 178B| 0 0 | 54 17
0 0 50 50 0 0| 0 232k| 128B 178B| 0 0 | 66 30
- ----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read writ| recv send| in out | int csw
0 0 45 55 0 0| 0 392k| 126B 178B| 0 0 | 85 22
0 0 50 50 0 0| 0 296k| 126B 322B| 0 0 | 64 10
0 0 65 35 0 0| 24k 472k| 448B 486B| 0 0 | 114 99
1 1 20 78 0 0| 120k 0 | 66B 178B| 0 0 | 86 96
0 0 49 51 0 0| 0 0 | 66B 178B| 0 0 | 9 12
0 1 64 36 0 0|2720k 456k| 276B 178B| 0 0 | 151 124
0 0 100 0 0 0| 0 0 | 196B 178B| 0 0 | 32 24
0 0 100 0 0 0| 0 0 | 66B 178B| 0 0 | 9 12
0 0 100 0 0 0| 0 0 | 66B 178B| 0 0 | 10 13
0 0 80 19 0 0|8192B 1248k| 66B 178B| 0 0 | 177 205
0 0 54 46 0 0| 16k 328k| 198B 486B| 0 0 | 99 111
0 0 53 47 0 0| 0 344k| 126B 178B| 0 0 | 82 78
0 0 31 69 0 0| 0 312k| 66B 178B| 0 0 | 79 101
0 0 60 40 0 0| 0 240k| 192B 372B| 0 0 | 76 89
0 0 24 76 0 0| 0 280k| 66B 178B| 0 0 | 71 81
0 0 47 53 0 0| 0 120k| 66B 178B| 0 0 | 36 40
0 0 50 50 0 0| 0 304k| 66B 178B| 0 0 | 79 99
0 0 56 44 0 0| 0 144k| 66B 178B| 0 0 | 42 84
0 0 50 50 0 0| 16k 136k| 186B 178B| 0 0 | 50 64
0 0 59 41 0 0| 0 328k| 192B 372B| 0 0 | 71 57
0 0 22 78 0 0| 0 280k| 66B 178B| 0 0 | 71 88
0 0 50 50 0 0| 0 256k| 66B 178B| 0 0 | 76 88
0 0 56 44 0 0| 24k 232k| 186B 178B| 0 0 | 80 102
- ----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read writ| recv send| in out | int csw
0 0 52 48 0 0| 0 240k| 132B 372B| 0 0 | 76 84
0 0 60 40 0 0| 0 416k| 186B 322B| 0 0 | 73 81
3 2 55 41 0 0|8192B 184k| 384B 1316B| 0 0 | 97 80
0 0 100 0 0 0| 0 0 | 428B 746B| 0 0 | 40 22
0 0 100 0 0 0| 0 0 | 246B 462B| 0 0 | 24 13
0 0 98 2 0 0| 0 2728k| 66B 178B| 0 0 | 22 21
0 0 100 0 0 0| 0 0 | 308B 462B| 0 0 | 23 11
0 0 100 0 0 0| 0 0 | 66B 178B| 0 0 | 23 17
0 0 100 0 0 0| 0 0 | 126B 178B| 0 0 | 12 11
0 0 100 0 0 0| 0 0 | 246B 462B| 0 0 | 23 15
0 0 99 1 0 0| 0 0 | 66B 178B| 0 0 | 10 14
0 0 99 1 0 0| 0 688k| 126B 178B| 0 0 | 110 15
0 0 100 0 0 0| 0 0 | 248B 178B| 0 0 | 21 15
0 0 100 0 0 0| 0 0 | 66B 178B| 0 0 | 10 13
0 0 100 0 0 0| 0 0 | 186B 178B| 0 0 | 21 13
0 0 100 0 0 0| 0 0 | 126B 178B| 0 0 | 14 17
0 0 100 0 0 0| 0 0 | 186B 462B| 0 0 | 20 11
0 0 100 0 0 0| 0 0 | 126B 178B| 0 0 | 22 19
0 0 100 0 0 0| 0 0 | 186B 178B| 0 0 | 17 13
0 0 100 0 0 0| 0 0 | 126B 178B| 0 0 | 12 14
0 0 100 0 0 0| 0 0 | 126B 178B| 0 0 | 13 13
0 0 100 0 0 0| 0 0 | 186B 178B| 0 0 | 13 12
0 0 100 0 0 0| 0 0 | 126B 178B| 0 0 | 11 11
- ----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read writ| recv send| in out | int csw
0 0 78 23 0 0| 0 32k| 126B 178B| 0 0 | 20 21
0 0 100 0 0 0| 0 0 | 126B 322B| 0 0 | 12 13
0 0 100 0 0 0| 0 0 | 66B 178B| 0 0 | 9 12
0 0 100 0 0 0| 0 0 | 66B 178B| 0 0 | 10 13
0 0 100 0 0 0| 0 0 | 126B 178B| 0 0 | 15 15
0 0 100 0 0 0| 0 0 | 126B 178B| 0 0 | 13 11
0 0 100 0 0 0| 0 0 | 126B 314B| 0 0 | 16 15
0 0 100 0 0 0| 0 0 | 186B 326B| 0 0 | 16 11
0 0 100 0 0 0| 0 112k| 126B 178B| 0 0 | 22 15
0 0 100 0 0 0| 0 0 | 186B 178B| 0 0 | 14 14
0 0 100 0 0 0| 0 0 | 66B 178B| 0 0 | 12 12
0 0 100 0 0 0| 0 0 | 126B 178B| 0 0 | 14 13
0 0 100 0 0 0| 0 0 | 66B 178B| 0 0 | 10 13
0 0 99 2 0 0| 0 8192B| 66B 178B| 0 0 | 12 15
0 0 100 0 0 0| 0 0 | 186B 178B| 0 0 | 21 14
0 0 100 0 0 0| 0 0 | 126B 178B| 0 0 | 15 13
0 0 100 0 0 0| 0 0 | 186B 178B| 0 0 | 16 15
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)
iD8DBQFLjMy5fg746kcGBOwRAs6KAJ9BeSRkvsIwt3/z/KYQcW6fIKxkHgCgjFaH
GDjMz//WUy7m2EAeD27HYpw=
=oK05
-----END PGP SIGNATURE-----
_______________________________________________
CentOS mailing list
CentOS@xxxxxxxxxx
http://lists.centos.org/mailman/listinfo/centos
