Peter Rabbitson wrote:
Hi,
I have been trying to figure out the best chunk size for raid10 before
migrating my server to it (currently raid1). I am looking at 3 offset
stripes, as I want to have two drive failure redundancy, and offset
striping is said to have the best write performance, with read
performance equal to far. Information on the internet is scarce so I
decided to test chunking myself. I used the script
http://rabbit.us/pool/misc/raid_test.txt to iterate through different
chunk sizes, and try to dd the resulting array to /dev/null. I
deliberately did not make a filesystem on top of the array - I was
just looking for raw performance, and since the FS layer is not
involved no caching/optimization is taking place. I also monitored the
process with dstat in a separate window, and memory usage confirmed
that this method is valid.
I got some pretty weird results:
http://rabbit.us/pool/misc/raid_test_results.txt
From all my readings so far I thought that with chunk size increase
the large block access throughput decreases while small block reads
increase, and it is just a matter of finding a "sweet spot" balancing
them out. The results, however, clearly show something else. There are
some inconsistencies, which I attribute to my non-scientific approach,
but the trend is clearly showing.
Here are the questions I have:
* Why did the test show best consistent performance over a 16k chunk?
Is there a way to determine this number without running a lengthy
benchmark, just from knowing the drive performance?
By any chance did you remember to increase stripe_cache_size to match
the chunk size? If not, there you go.
* Why although I have 3 identical chunks of data at any time, dstat
never showed simultaneous reading from more than 2 drives. Every dd
run was accompanied by maxing out one of the drives at 58MB/s and
another one was trying to catch up to various degrees depending on the
chunk size. Then on the next dd run two other drives would be
(seemingly random) selected and the process would repeat.
* What the test results don't show but dstat did is how the array
resync behaved after the array creation. Although my system can
sustain reads from all 4 drives at the max speed of 58MB/s, here is
what the resync at different chunk sizes looked like:
32k - simultaneous reads from all 4 drives at 47MB/s sustained
64k - simultaneous reads from all 4 drives at 56MB/s sustained
128k - simultaneous reads from all 4 drives at 54MB/s sustained
512k - simultaneous reads from all 4 drives at 30MB/s sustained
1024k - simultaneous reads from all 4 drives at 38MB/s sustained
4096k - simultaneous reads from all 4 drives at 44MB/s sustained
16384k - simultaneous reads from all 4 drives at 46MB/s sustained
32768k - simultaneous reads from 2 drives at 58MB/s sustained and
the other two at 26MB/s sustained alternating the speed
between the pairs of drives every 3 seconds or so
65536k - All 4 drives started at 58MB/s sustained gradually
reducing to 44MB/s sustained at the same time
I repeated just the creation of arrays - the results are consistent.
Is there any explanation for this?
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
bill davidsen <davidsen@xxxxxxx>
CTO TMR Associates, Inc
Doing interesting things with small computers since 1979
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
