On 09/14/2012 12:15 PM, Nick Couchman wrote:
While I'm talking about XFS... I know that RBD's use a default object
size of 4MB. I've stuck with that so far.. Would it be beneficial to
mount XFS with -o allocsize=4M ? What is the object size that gets
used for non-RBD cases -- i.e. just dumping objects into data pool?
Don't know about -o allocsize -- benchmark it!
...and let us know what you come up with! I'm also using XFS for the underlying filesystem on which CEPH runs (and using RBD), and would be really interested to know if changing the alloc size improves performance!
-Nick
Hi Guys,
There was a change 2.6.38 to the way that speculative preallocation
works that basically lets small writes behave like allocsize is not set,
and large writes behave like a large one is set:
http://permalink.gmane.org/gmane.comp.file-systems.xfs.general/38403
Having said that, I had my test gear all ready to go so I decided to
give it a try:
Setup:
- 1 node
- 6 OSDs with 7200rpm data disks.
- Journals on 2 Intel 520 SSDs (3 per SSD)
- LSI SAS2008 Controller (9211-8i)
- Network: Localhost
- Ceph 0.50
- Ubuntu 12.04
- Kernel 3.4
- XFS mkfs options: -f -i size=2048
- Common XFS mount options: -o noatime
- No replication
- 8 concurrent rados bench instances.
- 32 concurrent 4MB ops per instance (256 concurrent ops total)
Without allocsize=4M:
781.454MB/s
With allocsize=4M:
453.335MB/s
I'm guessing that it's perhaps slower as we've told XFS to optimize for
large files, but the metadata in /meta is very small, and we were
already getting benefits from the new speculative preallocation patches
that were introduced in 2.6.38 to combat fragmentation of the 4MB objects.
Mark
--------
This e-mail may contain confidential and privileged material for the sole use of the intended recipient. If this email is not intended for you, or you are not responsible for the delivery of this message to the intended recipient, please note that this message may contain SEAKR Engineering (SEAKR) Privileged/Proprietary Information. In such a case, you are strictly prohibited from downloading, photocopying, distributing or otherwise using this message, its contents or attachments in any way. If you have received this message in error, please notify us immediately by replying to this e-mail and delete the message from your mailbox. Information contained in this message that does not relate to the business of SEAKR is neither endorsed by nor attributable to SEAKR.
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
