Hi there!
We have two machines, both with LSI MPT Fusion SCSI cards in them. They
have the same CPUs, memory, etc. Their disk performance is quite
different however.
The two machines are "crusaders" and "minsk". They're both dual Operton
servers, with six 10k 72G SCA disks in them, arranged as software
RAID-10. RAM in both is the same, 4G. Both are running Debian sarge
AMD64.
Initally we dismissed the huge difference in hdparm -tT numbers as just
hdparm not really being a useful test, so we've run bonnie with default
settings, and there's still quite a difference between them:
Minsk
=====
Version 1.03 ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
minsk 8G 54409 98 147492 32 18438 5 25201 44 32090 6 601.4 1
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 5569 99 +++++ +++ +++++ +++ 5819 99 +++++ +++ 20239 99
/dev/md3:
Timing cached reads: 3432 MB in 2.00 seconds = 1715.40 MB/sec
Timing buffered disk reads: 224 MB in 3.02 seconds = 74.13 MB/sec
Crusaders
=========
Version 1.03 ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
crusaders 8G 52445 95 104509 19 21752 6 30754 54 86996 17 520.5 1
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 5824 99 +++++ +++ +++++ +++ 5917 99 +++++ +++ 19196 99
/dev/md1:
Timing cached reads: 3836 MB in 2.00 seconds = 1916.37 MB/sec
Timing buffered disk reads: 418 MB in 3.00 seconds = 139.17 MB/sec
The bit we're concerned about is the read speeds, block is way lower on
minsk than crusaders.
Kernel used is 2.6.12-rc3 with 3.03.00 patches from lsil.com applied.
I've tried various combinations of kernel, without any improvement.
We've also tried the kernel on crusaders, but performance was even worse
then (as in, around 3M/s, instead of 30 we get now..)
We suspect it's caused by a quirk of a newer version of the silicon on
the controller, they're slightly different in lspci:
Minsk
=====
0000:03:02.0 SCSI storage controller: LSI Logic / Symbios Logic 53c1030
PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 08)
Subsystem: LSI Logic / Symbios Logic: Unknown device 1060
Flags: bus master, 66MHz, medium devsel, latency 128, IRQ 30
I/O ports at 3000 [size=256]
Memory at fe120000 (64-bit, non-prefetchable) [size=128K]
Memory at fe100000 (64-bit, non-prefetchable) [size=128K]
Capabilities: [50] Power Management version 2
Capabilities: [58] Message Signalled Interrupts: 64bit+
Queue=0/0 Enable-
Capabilities: [68] PCI-X non-bridge device.
Crusaders
=========
0000:03:02.0 SCSI storage controller: LSI Logic / Symbios Logic 53c1030
PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 07)
Subsystem: LSI Logic / Symbios Logic: Unknown device 1060
Flags: bus master, 66MHz, medium devsel, latency 128, IRQ 30
I/O ports at 4000 [size=256]
Memory at fe120000 (64-bit, non-prefetchable) [size=128K]
Memory at fe100000 (64-bit, non-prefetchable) [size=128K]
Capabilities: [50] Power Management version 2
Capabilities: [58] Message Signalled Interrupts: 64bit+
Queue=0/0 Enable-
Capabilities: [68] PCI-X non-bridge device.
Has anyone else seen this effect, or aware of a possible cause?
Thanks!
--
David Zanetti <david.zanetti@xxxxxxxxxxxxxxx>
Team Leader, Systems Administration
Catalyst IT Limited
+64-4-8032233 +64-21-402260
Attachment:
signature.asc
Description: This is a digitally signed message part
