On 16-11-30 12:07:33, Zhongyan Gu wrote: > Jason, > I test Jewel and confirmed Jewel has no such issue. > Could you tell me what is the specific pull that can be backported to > hammer to fix this issue?? PR: https://github.com/ceph/ceph/pull/12218 > > Zhongyan > > On Tue, Nov 29, 2016 at 10:52 PM, Jason Dillaman <jdillama@xxxxxxxxxx> > wrote: > > > You are correct that there is an issue with the Hammer version when > > calculating diffs. If the clone has an object that obscures an extent > > within the parent image but didn't exist within the first snapshot of > > the clone, the diff results from the parent image won't be included in > > the result. I'll open a ticket against this issue, but in the meantime > > I'd suggest trying a Jewel-based librbd client instead of Hammer. > > > > On Mon, Nov 28, 2016 at 10:18 PM, Zhongyan Gu <zhongyan.gu@xxxxxxxxx> > > wrote: > > > Thank you Jason. > > > We are designing a backup system for production cluster based on ceph's > > > export /import diff feature. > > > We found this issue and hopefully it can be confirmed and then fixed > > soon. > > > If you need any more information for debugging, Please just let me know. > > > > > > Thanks, > > > Zhongyan > > > > > > On Mon, Nov 28, 2016 at 10:05 PM, Jason Dillaman <jdillama@xxxxxxxxxx> > > > wrote: > > >> > > >> OK, in that case, none of my previous explanation is relevant. I'll > > >> spin up a hammer cluster and try to reproduce. > > >> > > >> On Wed, Nov 23, 2016 at 9:13 PM, Zhongyan Gu <zhongyan.gu@xxxxxxxxx> > > >> wrote: > > >> > BTW, I used Hammer 0.94.5 to do the test. > > >> > > > >> > Zhongyan > > >> > > > >> > On Thu, Nov 24, 2016 at 10:07 AM, Zhongyan Gu <zhongyan.gu@xxxxxxxxx> > > >> > wrote: > > >> >> > > >> >> Thank you Jason. My test shows in the following case, image B will be > > >> >> exactly same: > > >> >> 1. clone image A from parent: > > >> >> #rbd clone 1124-parent@snap1 A > > >> >> > > >> >> 2. create snap for A > > >> >> #rbd snap create A@snap1 > > >> >> > > >> >> 3. create empty image B > > >> >> #rbd create B -s 1 > > >> >> > > >> >> 4. export-diff A then impor-diff B: > > >> >> #rbd export-diff A@snap1 -|./rbd import-diff - B > > >> >> > > >> >> 5. check A@snap1 equals B@snap1 > > >> >> #rbd export A@snap1 -|md5sum > > >> >> Exporting image: 100% complete...done. > > >> >> 880709d7352b6c9926beb1d829673366 - > > >> >> #rbd export B@snap1 -|md5sum > > >> >> Exporting image: 100% complete...done. > > >> >> 880709d7352b6c9926beb1d829673366 - > > >> >> output shows A@snap1 equals B@snap1 > > >> >> > > >> >> However, in the following case, image B will not be exactly same: > > >> >> 1. clone image A from parent: > > >> >> #rbd clone 1124-parent@snap1 A > > >> >> > > >> >> 2. create snap for A > > >> >> #rbd snap create A@snap1 > > >> >> > > >> >> 3. use fio make some change to A > > >> >> > > >> >> 4. create empty image B > > >> >> #rbd create B -s 1 > > >> >> > > >> >> 4. export-diff A then impor-diff B: > > >> >> #rbd export-diff A@snap1 -|./rbd import-diff - B > > >> >> > > >> >> 5. check A@snap1 equals B@snap1 > > >> >> #rbd export A@snap1 -|md5sum > > >> >> Exporting image: 100% complete...done. > > >> >> 880709d7352b6c9926beb1d829673366 - > > >> >> #rbd export B@snap1 -|md5sum > > >> >> Exporting image: 100% complete...done. > > >> >> bbf7cf69a84f3978c66f5eb082fb91ec - > > >> >> output shows A@snap1 DOES NOT equal B@snap1 > > >> >> > > >> >> The second case can always be reproduced. What is wrong with the > > second > > >> >> case? > > >> >> > > >> >> Thanks, > > >> >> Zhongyan > > >> >> > > >> >> > > >> >> On Wed, Nov 23, 2016 at 10:11 PM, Jason Dillaman < > > jdillama@xxxxxxxxxx> > > >> >> wrote: > > >> >>> > > >> >>> What you are seeing sounds like a side-effect of deep-flatten > > support. > > >> >>> If you write to an unallocated extent within a cloned image, the > > >> >>> associated object extent must be read from the parent image, > > modified, > > >> >>> and written to the clone image. > > >> >>> > > >> >>> Since the Infernalis release, this process has been tweaked if the > > >> >>> cloned image has a snapshot. In that case, the associated object > > >> >>> extent is still read from the parent, but instead of being modified > > >> >>> and written to the HEAD revision, it is left unmodified and is > > written > > >> >>> to "pre" snapshot history followed by writing the original > > >> >>> modification (w/o the parent's object extent data) to the HEAD > > >> >>> revision. > > >> >>> > > >> >>> This change to the IO path was made to support flattening clones and > > >> >>> dissociating them from their parents even if the clone had > > snapshots. > > >> >>> > > >> >>> Therefore, what you are seeing with export-diff is actually the > > >> >>> backing object extent of data from the parent image written to the > > >> >>> clone's "pre" snapshot history. If you had two snapshots and your > > >> >>> export-diff'ed from the first to second snapshot, you wouldn't see > > >> >>> this extra data. > > >> >>> > > >> >>> To your question about how to prepare image B to make sure it will > > be > > >> >>> exactly the same, the answer is that you don't need to do anything. > > In > > >> >>> your example above, I am assuming you are manually creating an empty > > >> >>> Image B and using "import-diff" to populate it. The difference in > > the > > >> >>> export-diff is most likely related to fact that the clone lost its > > >> >>> sparseness on any backing object that was written (e.g. instead of a > > >> >>> one or more 512 byte diffs within a backing object extent, you will > > >> >>> see a single, full-object extent with zeroes where the parent image > > >> >>> had no data). > > >> >>> > > >> >>> > > >> >>> On Wed, Nov 23, 2016 at 5:06 AM, Zhongyan Gu <zhongyan.gu@xxxxxxxxx > > > > > >> >>> wrote: > > >> >>> > Let me make the issue more clear. > > >> >>> > Suppose I cloned image A from a parent image and create snap1 for > > >> >>> > image > > >> >>> > A > > >> >>> > and then make some change of image A. > > >> >>> > If I did the rbd export-diff @snap1. how should I prepare the > > >> >>> > existing > > >> >>> > image > > >> >>> > B to make sure it will be exactly same with image A@snap1 after > > >> >>> > import-diff > > >> >>> > against this image B. > > >> >>> > > > >> >>> > Thanks, > > >> >>> > Zhongyan > > >> >>> > > > >> >>> > > > >> >>> > On Wed, Nov 23, 2016 at 11:34 AM, Zhongyan Gu > > >> >>> > <zhongyan.gu@xxxxxxxxx> > > >> >>> > wrote: > > >> >>> >> > > >> >>> >> Thanks Jason, very clear explanation. > > >> >>> >> However, I found some strange behavior when export-diff on a > > cloned > > >> >>> >> image, > > >> >>> >> not sure it is a bug on calc_snap_set_diff(). > > >> >>> >> The test is, > > >> >>> >> Image A is cloned from a parent image. then create snap1 for > > image > > >> >>> >> A. > > >> >>> >> The content of export-diff A@snap1 will be changed when update > > >> >>> >> image > > >> >>> >> A. > > >> >>> >> Only after image A has no overlap with parent, the content of > > >> >>> >> export-diff > > >> >>> >> A@snap1 is stabled, which is almost zero. > > >> >>> >> I don't think it is a designed behavior. export-diff A@snap1 > > should > > >> >>> >> always > > >> >>> >> get a stable output no matter image A is cloned or not. > > >> >>> >> > > >> >>> >> Please correct me if anything wrong. > > >> >>> >> > > >> >>> >> Thanks, > > >> >>> >> Zhongyan > > >> >>> >> > > >> >>> >> > > >> >>> >> > > >> >>> >> > > >> >>> >> On Tue, Nov 22, 2016 at 10:31 PM, Jason Dillaman > > >> >>> >> <jdillama@xxxxxxxxxx> > > >> >>> >> wrote: > > >> >>> >>> > > >> >>> >>> On Tue, Nov 22, 2016 at 5:31 AM, Zhongyan Gu > > >> >>> >>> <zhongyan.gu@xxxxxxxxx> > > >> >>> >>> wrote: > > >> >>> >>> > So if initial snapshot is NOT specified, then: > > >> >>> >>> > rbd export-diff image@snap1 will diff all data to snap1. this > > >> >>> >>> > cmd > > >> >>> >>> > equals to > > >> >>> >>> > : > > >> >>> >>> > rbd export image@snap1. Is my understand right or not?? > > >> >>> >>> > > >> >>> >>> > > >> >>> >>> While they will both export all data associated w/ image@snap1, > > >> >>> >>> the > > >> >>> >>> "export" command will generate a raw, non-sparse dump of the > > full > > >> >>> >>> image whereas "export-diff" will export only sections of the > > image > > >> >>> >>> that contain data. The file generated from "export" can be used > > >> >>> >>> with > > >> >>> >>> the "import" command to create a new image, whereas the file > > >> >>> >>> generated > > >> >>> >>> from "export-diff" can only be used with "import-diff" against > > an > > >> >>> >>> existing image. > > >> >>> >>> > > >> >>> >>> -- > > >> >>> >>> Jason > > >> >>> >> > > >> >>> >> > > >> >>> > > > >> >>> > > >> >>> > > >> >>> > > >> >>> -- > > >> >>> Jason > > >> >> > > >> >> > > >> > > > >> > > >> > > >> > > >> -- > > >> Jason > > > > > > > > > > > > > > -- > > Jason > > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |