AFR had similar problem 2 years back and we started persisting afr children and graph always gets generated with what the xattrs will be in that order as an option. You can check afr_pending_xattrs_init() for it. On the glusterd side we give an identifer for each of the bricks.
The main reason we did this is because we didn't want to change this logic if new xlators get introduced in between client xlator and afr. What will you do if you add one more xlator between readdir-ahead and dht in future?
The main reason we did this is because we didn't want to change this logic if new xlators get introduced in between client xlator and afr. What will you do if you add one more xlator between readdir-ahead and dht in future?
Now that I am seeing this mail and also Facebook guys also expressed that there should be a way for gluster to identify if it is talking to right bricks recently when they met us. May be we should let each xlator choose an identifier for its subvolumes and persist it on the bricks and use that instead of depending on mgmt to do the right things.
I understand this patch is already solving your problem. I see that it is merged as well but it is not future proof.
On Wed, May 17, 2017 at 10:03 AM, Raghavendra Gowdappa <rgowdapp@xxxxxxxxxx> wrote:
dht: Add readdir-ahead in rebalance graph if parallel-readdir is on
Issue:
The value of linkto xattr is generally the name of the dht's
next subvol, this requires that the next subvol of dht is not
changed for the life time of the volume. But with parallel
readdir enabled, the readdir-ahead loaded below dht, is optional.
The linkto xattr for first subvol, when:
- parallel readdir is enabled : "<volname>-readdir-head-0"
- plain distribute volume : "<volname>-client-0"
- distribute replicate volume : "<volname>-afr-0"
The value of linkto xattr is "<volname>-readdir-head-0" when
parallel readdir is enabled, and is "<volname>-client-0" if
its disabled. But the dht_lookup takes care of healing if it
cannot identify which linkto subvol, the xattr points to.
In dht_lookup_cbk, if linkto xattr is found to be "<volname>-client-0"
and parallel readdir is enabled, then it cannot understand the
value "<volname>-client-0" as it expects "<volname>-readdir-head-0".
In that case, dht_lookup_everywhere is issued and then the linkto file
is unlinked and recreated with the right linkto xattr. The issue is
when parallel readdir is enabled, mount point accesses the file
that is currently being migrated. Since rebalance process doesn't
have parallel-readdir feature, it expects "<volname>-client-0"
where as mount expects "<volname>-readdir-head-0". Thus at some point
either the mount or rebalance will fail.
Solution:
Enable parallel-readdir for rebalance as well and then do not
allow enabling/disabling parallel-readdir if rebalance is in
progress.
Change-Id: I241ab966bdd850e667f7768840540546f5289483
BUG: 1436090
Signed-off-by: Poornima G <pgurusid@xxxxxxxxxx>
Reviewed-on: https://review.gluster.org/17056
Smoke: Gluster Build System <jenkins@xxxxxxxxxxxxxxxxx>
NetBSD-regression: NetBSD Build System <jenkins@xxxxxxxxxxxxxxxxx>
CentOS-regression: Gluster Build System <jenkins@xxxxxxxxxxxxxxxxx>
Patch: https://review.gluster.org/17056
owners: Me and Poornima
PS: readdir-ahead is loaded in rebalance graph on top of each subvolume of DHT.
regards,
Raghavendra
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://lists.gluster.org/mailman/listinfo/gluster-devel
--
Pranith
_______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx http://lists.gluster.org/mailman/listinfo/gluster-devel
