Hi,
As part of Bit-rot detection feature a file that has its data changed
due to some backend errors is marked as a bad file by the scrubber (sets
an extended attribute indicating its a bad file). Now, the access to the
bad file has to be denied (to prevent wrong data being served).
In bit-rot-stub xlator (the xlator which does object versioning and
sends notifications to BitD upon object modification) the check for
whether the file is bad or not can be done in lookup where if the xattr
is set, then the object can be marked as bad within its inode context as
well. But the problem is what if the object was not marked as bad at the
time of lookup and later it was marked bad. Now when a fop such as open,
readv or writev comes, the fops should not be allowed. If its fuse
client from which the file is being accessed, then probably its ok to
rely only on lookups (to check if its bad or not), as fuse sends lookups
before sending fops. But for NFS, once the lookup is done and filehandle
is available further lookups are not sent. In that case relying only on
lookup to check if its a bad file or not is suffecient.
Below 3 solutions in bit-rot-stub xlator seem to address the above issue.
1) Whenever a fop such as open, readv or writev comes, check in the
inode context if its a bad file or not. If not, then send a getxattr of
bad file xattr on that file. If its present, then set the bad file
attribute in the inode context and fail the fop.
But for above operation, a getxattr call has to be sent downwards for
almost each open or readv or writev. If the file is identified as bad,
then getxattr might not be necessary. But for good files extra getxattr
might affect the performance.
OR
2) Set a key in xdata whenever open, readv, or writev comes (in
bit-rot-stub xlator) and send it downwards. The posix xlator can look
into the xdata and if the key for bad file identification is present,
then it can do getxattr as part of open or readv or writev itself and
send the response back in xdata itself.
Not sure whether the above method is ok or not as it overloads open,
readv and writev. Apart from that, the getxattr disk operation is still
done.
OR
3) Once the file is identified as bad, the scrubber marks it as bad (via
setxattr) by sending a call to to bit-rot-stub xlator. Bit-rot-stub
xlator marks the file as bad in the inode context once it receives the
notification from scrubber that a file is bad. This saves those getxattr
calls being made from other fops (either in bit-rot-stub xlator or posix
xlator).
But the trick is what if the inode gets forgotten or the brick restarts.
But I think in that case, checking in lookup call is suffecient (as in
both inode forgets and brick restarts, a lookup will definitely come if
there is an accss to that file).
Please provide feedback on above 3 methods. If there are any other
solutions which might solve this issue, they are welcome.
Regards,
Raghavendra Bhat
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-devel
