On Wed, Mar 02, 2016 at 11:43:50AM +0530, Venky Shankar wrote: > On Sat, Feb 27, 2016 at 06:32:38AM -0500, Joseph Fernandes wrote: > > Yep Agree! :) > > > > Lets hear from the bitrot folks, what they have to propose. > > Apologies for late reply. > > > > > ~Joe > > > > ----- Original Message ----- > > From: "Niels de Vos" <ndevos@xxxxxxxxxx> > > To: "Joseph Fernandes" <josferna@xxxxxxxxxx> > > Cc: "Gluster Devel" <gluster-devel@xxxxxxxxxxx> > > Sent: Saturday, February 27, 2016 4:28:43 PM > > Subject: Re: Bitrot/Tering : Bad files get migrated and hence corruption goes undetected. > > > > On Fri, Feb 26, 2016 at 11:01:28PM -0500, Joseph Fernandes wrote: > > > Well correctly we dont migrate the existing signature, the file starts > > > it life fresh in the new tier(i.e get the bit rot version 1 on the new > > > tier), > > > Now this is also the case with any special xattr/attributes of the > > > file. > > > Again we rely heavily on the dht rebalance mechanism for migrations, > > > which also doesnt carry special attributes/xattr. > > > > Is there a good reason to not migrate the bitrot signature? Relying on > > an existing functionality is fine, but if it does not address all your > > needs, you have a valid use-case to improve it. > > That could be done. However, an I/O operation on the object during migration > should invalidate the signature and the object should be signed again. > > AFAICS, there needs to be some infrastructure to avoid (re)signing of an > object if it's fresh after migration. [Replying to my own mail] What's needed here is an hint that one or more open()'s happened on the object in the migration window. In that case, the object version needs to be incremented followed by a (re)sign after last close. The other part that needs thinking is to scrub the object upon access (if it's atleast signed). > > Thoughts? > > > > > Niels > > > > > > > > > > > ----- Original Message ----- > > > From: "Niels de Vos" <ndevos@xxxxxxxxxx> > > > To: "Joseph Fernandes" <josferna@xxxxxxxxxx> > > > Cc: "Gluster Devel" <gluster-devel@xxxxxxxxxxx> > > > Sent: Friday, February 26, 2016 10:33:11 PM > > > Subject: Re: Bitrot/Tering : Bad files get migrated and hence corruption goes undetected. > > > > > > On Fri, Feb 26, 2016 at 09:32:46AM -0500, Joseph Fernandes wrote: > > > > Hi All, > > > > > > > > This is a discussion mail on the following issue, > > > > > > > > 1. Object is corrupted before it could be signed: In this case, the corrupted > > > > object is signed and get migrated upon I/O. There's no way to identify corruption > > > > for this set of objects. > > > > > > > > 2. Object is signed (but not scrubbed) and corruption happens thereafter: > > > > In this case, as of now, integrity checking is not done on the fly > > > > and the object would get migrated (and signed again in the hot tier). > > > > > > > > > > > > The (1) is definitely not a issue with bitrot with tiering. But (2) we can do something to avoid > > > > corrupted file from getting migrated. Before we migrate files we can scrub it, but its just a naive > > > > thought, any better suggestions? > > > > > > Is there a reason the existing signature can not be migrated? Why does > > > it become invalid? > > > > > > Niels > > _______________________________________________ > > Gluster-devel mailing list > > Gluster-devel@xxxxxxxxxxx > > http://www.gluster.org/mailman/listinfo/gluster-devel _______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-devel
