On 06/04/2012 07:14 PM, Ole Tange wrote:
But I cannot explain why even the best performance (4600 MB/11s = 420
MB/s) is not even close to the checksum performance reported by the
kernel at boot (6196 MB/s):
Mar 13 16:02:42 server kernel: [ 35.120035] raid6: using
algorithm sse2x4 (6196 MB/s)
Can you explain why I only get 420 MB/s of real world checksumming
instead of 6196 MB/s?
In the best possible case, you would get 22x 1-disk bandwidth, which
would be ~120MB/s (assuming RAID6, and "infinite" speed of checksum
computation). This is your "theoretical" upper bound on performance.
Your pragmatic upper bound on performance is reduced from the
theoretical by many issues, including various hardware issues
(controller, PCIe lanes, memory, disk, ...), as well as software (IO
stack traversal, elevators, buffers/cache fills, ... etc.).
Aside from this, it is very rare that you will have a single application
reading and writing at full stripe width all the time, which would be
the optimal case for you. There are some, and we've play with a number
for our customers. But they are the exception and not the rule.
Your real world IO performance for 24 disks is 420MB/s. So this
particular setup is, from your numbers, operating at about 16% peak
efficiency per disk +/- some. This is not uncommon for people's
self-built systems. The checksumming isn't your rate limiting feature.
Other things are.
Joe
/Ole
--
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
email: landman@xxxxxxxxxxxxxxxxxxxxxxx
web : http://scalableinformatics.com
http://scalableinformatics.com/sicluster
phone: +1 734 786 8423 x121
fax : +1 866 888 3112
cell : +1 734 612 4615
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
