On 03/05/2013 08:33 PM, Sage Weil wrote:
On Tue, 5 Mar 2013, Wido den Hollander wrote:
Wido, by 'user quota' do you mean something that is uid-based, or would
enforcement on subtree/directory quotas be sufficient for your use cases?
I've been holding out hope that uid-based usage accounting is a thing of
the past and that subtrees are sufficient for real users... in which case
adding enfocement to the existing rstats infrastructure is a very
manageable task.
I mean actual uid-based quotas. That still plays nice with shared environments
like Samba or so where you have all homedirectories on a shared filesystems
and you set per user quotas. Samba reads out those quotas and propagates them
to the (Windows) client.
Does samba propagate the quota information (how much space is
used/available) or do enforcement on the client side? (Is client
enforcement even necessary/useful if the backend will stop writes when the
quota is exceeded?)
I'm not sure. It will at least tell the user how much he/she is using on
that volume and what the quota is. Not sure who enforces, Samba or the
filesystem.
From a quick Google it seems like the filesystem has to enforce the
quota, Samba doesn't.
I know this was a problem with ZFS as well. They also said they could do "per
filesystem quotas" so that would be sufficient, but for example NFS doesn't
export filesystems mounted in a export, so if you have a bunch of
homedirectories on the filesystem and you want to account the usage of each
user it's getting kind of hard.
This could be solved if the clients directly mounted CephFS though.
I'm talking about setups where you have 100k users in a LDAP and they all have
their data in a single filesystem and you want to track the usage of each
user, that's not an easy task without uid-based quotas.
Wouldn't each user live in a sub- or home directory? If so, it seems like
the existing rstats would be sufficient to do the accounting piece; only
enforcement is missing.
Running 'du' on each directory would be much faster with Ceph since it
accounts tracks the subdirectories and shows their total size with an 'ls
-al'.
Environments with 100k users also tend to be very dynamic with adding and
removing users all the time, so creating separate filesystems for them would
be very time consuming.
Now, I'm not talking about enforcing soft or hard quotas, I'm just talking
about knowing how much space uid X and Y consume on the filesystem.
The part I'm most unclear on is what use cases people have where uid X and
Y are spread around the file system (not in a single or small set of sub
directories) and per-user (not, say, per-project) quotas are still
necessary. In most environments, users get their own home directory and
everything lives there...
I see a POSIX-filesystem as being partially legacy and a part of that
legacy is user quotas.
If you want existing applications who rely on userquotas to seamlessly
switch from NFS to CephFS they will need this to work.
We only talked about userquotas, but groupquotas are just as important.
If you have 10 users where 5 of them are in the group "webdev" and you
wan't to know how much space is being used by the group "webdev" you
want to probe the group quotas and you are done.
In some setups like we have users have data in different directories
outside their home directories / NFS exports. On one machine you just
run "quota -u <uid>" and you know how much user X is using spread out
over all the filesystems.
With rstats you would be able to achieve the same with some scripting,
but that doesn't make the migration seamless.
Wido
sage
Wido
sage
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Wido den Hollander
42on B.V.
Phone: +31 (0)20 700 9902
Skype: contact42on
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Wido den Hollander
42on B.V.
Phone: +31 (0)20 700 9902
Skype: contact42on
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
