Shyam wrote:
On 12/19/2015 01:13 AM, Venky Shankar wrote:
Shyam wrote:
<first off the cuff response>
On 12/18/2015 04:00 AM, Vijaikumar Mallikarjuna wrote:
Hi All,
Here is the summary of discussion we had today on Quota-v2
Project, user and group quotas will use the same logic of accounting
the
usage
Quota-v2 should be compatible with both DHT-v1 and DHT-v2
Project quotas will use gfid of the given path as the project-id.
each inode will have a associated project-id which needs to be
embedded within inode.
when creating new file, it will inherit project-id from its parent
inode.
In case if parent inode doesn't contain project-id, then there
should be a mechanism to
find the project-id given a gfid, so back-pointers can solve this
problem
Why would the parent not contain a project-id? and if it does not
contain a project-id, why would you need to find a project-id?
Updating project-id's for all children for a given directory on which a
project-id is set would required scanning. If we could figure
out the project-id for a given inode using back-pointers (closest
ancestor for which a project-id is set) then we'd not need the crawl and
update.
For existing directories when a quota is set, we need to do the scan and
set/update the quota information, right? This is unavoidable as far as I
can see.
I was more into trying to save the sub-tree update when a project-id is
set on a directory. If (with current dht) we could figure out the
project-id for the closest project-id set ancestor using back-pointers
(or whatever..) then we could possibly avoid the scan + update.
For a new entry, as responded in the other mail on this thread, if the
parent is not yet marked with a project-id then the child will inherit
the same when the scanner updates the project-id for the parent, till
then there is no need to account.
With DHT2, such a mechanism would query each MDS. For current DHT, it
might still be good enough.
With DHT2 the scanner (for updating quota information for pre-existing
directories) may need to split the work up and perform the operation. A
crude thought at the moment would be,
- Set the project-id for the grandparent where the quota enforcement starts
- Set the pj-id for the leaf objects for this directory, and set the
pj-id for the directories under this grandparent, but take up marking
leaves of these sub-directories as tasks in the other MDS stack
(wherever they belong, i.e same MDS or a different one).
What I can visualize is that, the parent is not yet part of a quota
project, hence it does not contain a project-id. When this becomes a
part of some quota restriction it would start getting a quota. This
seems to be one case where the project-id would be missing, but by
choice.
What am I missing here?
which xlator DHT/marker will set the project-id in the inode? this
needs to be decided
User and group quotas will use uid and gid to account the usage
Quota has below set of operation which should be performed as a single
transaction
read current size
read current contribution
update delta to the contribution
update size to user/project inode
In the current directory quota, to make this transaction crash
consistent we use a mechanism of setting dirty flag in a parent inode.
This mechanism will not work efficiently with project quotas.
Journaling
will be the good solution for crash-consistency.
Quota 2 depends on:
back-pointers: to find project-id from gfid
journaling: for crash consistency of accounting
Thanks,
Vijay
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-devel
