Hi guys,
As a distributed filesystem, all clients of CephFS share the whole
cluster's resources, for example, IOPS, throughput. In some cases,
resources will be occupied by some clients. So QoS for CephFS is
needed in most cases.
Based on the token bucket algorithm, I implement QoS for CephFS.
The basic idea is as follows:
Set QoS info as one of the dir's xattrs;
All clients can access the same dirs with the same QoS setting.
Similar to the Quota's config flow. when the MDS receives the QoS
setting, it'll also broadcast the message to all clients.
We can change the limit online.
And we will config QoS as follows, it supports
{limit/burst}{iops/bps/read_iops/read_bps/write_iops/write_bps}
configure setting, some examples:
setfattr -n ceph.qos.limit.iops -v 200 /mnt/cephfs/testdirs/
setfattr -n ceph.qos.burst.read_bps -v 200 /mnt/cephfs/testdirs/
getfattr -n ceph.qos.limit.iops /mnt/cephfs/testdirs/
getfattr -n ceph.qos /mnt/cephfs/testdirs/
But, there is also a big problem. For the bps{bps/write_bps/read_bps}
setting, if the bps is lower than the request's block size, the client
will be blocked until it gets enough token.
Any suggestion will be appreciated, thanks!
PR: https://github.com/ceph/ceph/pull/29266
