We have tracked the performance drop to the attached two commits in 2.6.28.6. The performance never fully recovers in later kernels so I presuming that the change in the write cache is still affecting MD today. The problem for us is that although we have slowly tracked it down, we have no understanding of linux at this level and simply wouldn’t know where go from this point. Considering this seems to only effect MD and not hardware based RAID (in our tests) I thought that this would be an appropriate place to post these patches and findings. There are 2 patches which impact MD performance via a filesystem: a) commit 66c85494570396661479ba51e17964b2c82b6f39 - write-back: fix nr_to_write counter b) commit fa76ac6cbeb58256cf7de97a75d5d7f838a80b32 - Fix page writeback thinko, causing Berkeley DB slowdown 1) no patches applied into 2.6.28.5 kernel: write speed is 1.1 GB/s via xfs 2) both patches are applied into 2.6.28.5 kernel: xfs drops to circa: 680 MB/s (like in kernel 2.6.28.6 and later) 3) put only one patch: 66c85494570396661479ba51e17964b2c82b6f39 (write-back: fix nr_to_write counter) - performance goes down to circa 780 MB/s 4) put only one patch: fa76ac6cbeb58256cf7de97a75d5d7f838a80b32 (Fix page writeback thinko) - the performance is good: 1.1 GB/s (on XFS) change log for 28.6 ftp://ftp.kernel.org/pub/linux/kernel/v2.6/ChangeLog-2.6.28.6 Hopefully this helps to resolve this.... Mark 2009/10/17 Mattias Hellström <hellstrom.mattias@xxxxxxxxx>: > (If I were you) I would further test the revisions between the > following and then look at the changelog for the culprit. Looks like > versions after this are just trying to regain the missing speed. > > Linux linux-tlfp 2.6.27.14-vanilla #1 SMP Fri Oct 16 00:56:25 BST 2009 > x86_64 x86_64 x86_64 GNU/Linux > > RAW: 1.1 > XFS 1.1 > > Linux linux-tlfp 2.6.27.20-vanilla #1 SMP Thu Oct 15 23:59:32 BST 2009 > x86_64 x86_64 x86_64 GNU/Linux > > RAw 1.1 GB/s > XFS: 487 MB/s >
Attachment:
fa76ac6cbeb58256cf7de97a75d5d7f838a80b32.patch
Description: Binary data
Attachment:
66c85494570396661479ba51e17964b2c82b6f39.patch
Description: Binary data
