On 09/14/2012 01:56 PM, Nick Couchman wrote:
Hi Guys,
There was a change 2.6.38 to the way that speculative preallocation
works that basically lets small writes behave like allocsize is not set,
and large writes behave like a large one is set:
http://permalink.gmane.org/gmane.comp.file-systems.xfs.general/38403
Having said that, I had my test gear all ready to go so I decided to
give it a try:
Setup:
- 1 node
- 6 OSDs with 7200rpm data disks.
- Journals on 2 Intel 520 SSDs (3 per SSD)
- LSI SAS2008 Controller (9211-8i)
- Network: Localhost
- Ceph 0.50
- Ubuntu 12.04
- Kernel 3.4
- XFS mkfs options: -f -i size=2048
- Common XFS mount options: -o noatime
- No replication
- 8 concurrent rados bench instances.
- 32 concurrent 4MB ops per instance (256 concurrent ops total)
Without allocsize=4M:
781.454MB/s
With allocsize=4M:
453.335MB/s
I'm guessing that it's perhaps slower as we've told XFS to optimize for
large files, but the metadata in /meta is very small, and we were
already getting benefits from the new speculative preallocation patches
that were introduced in 2.6.38 to combat fragmentation of the 4MB objects.
Mark
Interesting, thanks for the results, Mark. So, I guess don't tune unless you have a very good reason to do so? Or, if you're really going to try to squeeze all the performance possible, put your metadata on a separate FS with a different alloc size (or no alloc size specified) so that metadata access isn't adversely impacted by trying to tune data access?
-Nick
Well, the XFS guys certainly suggest default tuning in most cases... :)
http://xfs.org/index.php/XFS_FAQ#Q:_I_want_to_tune_my_XFS_filesystems_for_.3Csomething.3E
I think there is value in investigating things when you suspect a
problem though!
We've tried putting the meta directory on alternate partitions (note:
this isn't a good idea with btrfs). It hasn't really done much in some
of the tests we've done, but we weren't looking at testing this specific
scenario.
I think the bigger question is, what problem are you trying to solve?
Are you noticing lots of fragmentation? Slow performance with 4MB
writes? slow performance with small IO?
--------
This e-mail may contain confidential and privileged material for the sole use of the intended recipient. If this email is not intended for you, or you are not responsible for the delivery of this message to the intended recipient, please note that this message may contain SEAKR Engineering (SEAKR) Privileged/Proprietary Information. In such a case, you are strictly prohibited from downloading, photocopying, distributing or otherwise using this message, its contents or attachments in any way. If you have received this message in error, please notify us immediately by replying to this e-mail and delete the message from your mailbox. Information contained in this message that does not relate to the business of SEAKR is neither endorsed by nor attributable to SEAKR.
Mark
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
