On 10/01/2012 11:47 AM, Tommi Virtanen wrote:
On Thu, Sep 27, 2012 at 11:04 AM, Gregory Farnum<greg@xxxxxxxxxxx> wrote:
However, my suspicion is that you're limited by metadata throughput
here. How large are your files? There might be some MDS or client
tunables we can adjust, but rsync's workload is a known weak spot for
CephFS.
I feel like people are missing this part of Greg's message. Everyone
is so busy benchmarking RADOS small I/O, but what if it's currently
bottlenecked by all the file-level access operations that interact
with the MDS? Rsync causes a ton of those.
If you want to benchmark just the small IO, you can't compare rsync to rsync.
If you want to benchmark just the metadata part, rsync with 0-size
files might actually be an interesting workload.
I guess most of the small IO testing we've seen/done has been without
CephFS at all. It's entirely possible that the MDS is slowing things
down with an rsync workload like this on a fresh filesystem though.
Having said that, I don't like the way that our small IO performance
behaves (especially over time) when doing something like RADOS Bench.
It definitely seems like there is some pretty nasty underlying
filesystem metadata fragmentation or something going on after a while.
Mark
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
