Jason,
I test Jewel and confirmed Jewel has no such issue.
Could you tell me what is the specific pull that can be backported to hammer to fix this issue??
Zhongyan
On Tue, Nov 29, 2016 at 10:52 PM, Jason Dillaman <jdillama@xxxxxxxxxx> wrote:
You are correct that there is an issue with the Hammer version when
calculating diffs. If the clone has an object that obscures an extent
within the parent image but didn't exist within the first snapshot of
the clone, the diff results from the parent image won't be included in
the result. I'll open a ticket against this issue, but in the meantime
I'd suggest trying a Jewel-based librbd client instead of Hammer.
--
On Mon, Nov 28, 2016 at 10:18 PM, Zhongyan Gu <zhongyan.gu@xxxxxxxxx> wrote:
> Thank you Jason.
> We are designing a backup system for production cluster based on ceph's
> export /import diff feature.
> We found this issue and hopefully it can be confirmed and then fixed soon.
> If you need any more information for debugging, Please just let me know.
>
> Thanks,
> Zhongyan
>
> On Mon, Nov 28, 2016 at 10:05 PM, Jason Dillaman <jdillama@xxxxxxxxxx>
> wrote:
>>
>> OK, in that case, none of my previous explanation is relevant. I'll
>> spin up a hammer cluster and try to reproduce.
>>
>> On Wed, Nov 23, 2016 at 9:13 PM, Zhongyan Gu <zhongyan.gu@xxxxxxxxx>
>> wrote:
>> > BTW, I used Hammer 0.94.5 to do the test.
>> >
>> > Zhongyan
>> >
>> > On Thu, Nov 24, 2016 at 10:07 AM, Zhongyan Gu <zhongyan.gu@xxxxxxxxx>
>> > wrote:
>> >>
>> >> Thank you Jason. My test shows in the following case, image B will be
>> >> exactly same:
>> >> 1. clone image A from parent:
>> >> #rbd clone 1124-parent@snap1 A
>> >>
>> >> 2. create snap for A
>> >> #rbd snap create A@snap1
>> >>
>> >> 3. create empty image B
>> >> #rbd create B -s 1
>> >>
>> >> 4. export-diff A then impor-diff B:
>> >> #rbd export-diff A@snap1 -|./rbd import-diff - B
>> >>
>> >> 5. check A@snap1 equals B@snap1
>> >> #rbd export A@snap1 -|md5sum
>> >> Exporting image: 100% complete...done.
>> >> 880709d7352b6c9926beb1d829673366 -
>> >> #rbd export B@snap1 -|md5sum
>> >> Exporting image: 100% complete...done.
>> >> 880709d7352b6c9926beb1d829673366 -
>> >> output shows A@snap1 equals B@snap1
>> >>
>> >> However, in the following case, image B will not be exactly same:
>> >> 1. clone image A from parent:
>> >> #rbd clone 1124-parent@snap1 A
>> >>
>> >> 2. create snap for A
>> >> #rbd snap create A@snap1
>> >>
>> >> 3. use fio make some change to A
>> >>
>> >> 4. create empty image B
>> >> #rbd create B -s 1
>> >>
>> >> 4. export-diff A then impor-diff B:
>> >> #rbd export-diff A@snap1 -|./rbd import-diff - B
>> >>
>> >> 5. check A@snap1 equals B@snap1
>> >> #rbd export A@snap1 -|md5sum
>> >> Exporting image: 100% complete...done.
>> >> 880709d7352b6c9926beb1d829673366 -
>> >> #rbd export B@snap1 -|md5sum
>> >> Exporting image: 100% complete...done.
>> >> bbf7cf69a84f3978c66f5eb082fb91ec -
>> >> output shows A@snap1 DOES NOT equal B@snap1
>> >>
>> >> The second case can always be reproduced. What is wrong with the second
>> >> case?
>> >>
>> >> Thanks,
>> >> Zhongyan
>> >>
>> >>
>> >> On Wed, Nov 23, 2016 at 10:11 PM, Jason Dillaman <jdillama@xxxxxxxxxx>
>> >> wrote:
>> >>>
>> >>> What you are seeing sounds like a side-effect of deep-flatten support.
>> >>> If you write to an unallocated extent within a cloned image, the
>> >>> associated object extent must be read from the parent image, modified,
>> >>> and written to the clone image.
>> >>>
>> >>> Since the Infernalis release, this process has been tweaked if the
>> >>> cloned image has a snapshot. In that case, the associated object
>> >>> extent is still read from the parent, but instead of being modified
>> >>> and written to the HEAD revision, it is left unmodified and is written
>> >>> to "pre" snapshot history followed by writing the original
>> >>> modification (w/o the parent's object extent data) to the HEAD
>> >>> revision.
>> >>>
>> >>> This change to the IO path was made to support flattening clones and
>> >>> dissociating them from their parents even if the clone had snapshots.
>> >>>
>> >>> Therefore, what you are seeing with export-diff is actually the
>> >>> backing object extent of data from the parent image written to the
>> >>> clone's "pre" snapshot history. If you had two snapshots and your
>> >>> export-diff'ed from the first to second snapshot, you wouldn't see
>> >>> this extra data.
>> >>>
>> >>> To your question about how to prepare image B to make sure it will be
>> >>> exactly the same, the answer is that you don't need to do anything. In
>> >>> your example above, I am assuming you are manually creating an empty
>> >>> Image B and using "import-diff" to populate it. The difference in the
>> >>> export-diff is most likely related to fact that the clone lost its
>> >>> sparseness on any backing object that was written (e.g. instead of a
>> >>> one or more 512 byte diffs within a backing object extent, you will
>> >>> see a single, full-object extent with zeroes where the parent image
>> >>> had no data).
>> >>>
>> >>>
>> >>> On Wed, Nov 23, 2016 at 5:06 AM, Zhongyan Gu <zhongyan.gu@xxxxxxxxx>
>> >>> wrote:
>> >>> > Let me make the issue more clear.
>> >>> > Suppose I cloned image A from a parent image and create snap1 for
>> >>> > image
>> >>> > A
>> >>> > and then make some change of image A.
>> >>> > If I did the rbd export-diff @snap1. how should I prepare the
>> >>> > existing
>> >>> > image
>> >>> > B to make sure it will be exactly same with image A@snap1 after
>> >>> > import-diff
>> >>> > against this image B.
>> >>> >
>> >>> > Thanks,
>> >>> > Zhongyan
>> >>> >
>> >>> >
>> >>> > On Wed, Nov 23, 2016 at 11:34 AM, Zhongyan Gu
>> >>> > <zhongyan.gu@xxxxxxxxx>
>> >>> > wrote:
>> >>> >>
>> >>> >> Thanks Jason, very clear explanation.
>> >>> >> However, I found some strange behavior when export-diff on a cloned
>> >>> >> image,
>> >>> >> not sure it is a bug on calc_snap_set_diff().
>> >>> >> The test is,
>> >>> >> Image A is cloned from a parent image. then create snap1 for image
>> >>> >> A.
>> >>> >> The content of export-diff A@snap1 will be changed when update
>> >>> >> image
>> >>> >> A.
>> >>> >> Only after image A has no overlap with parent, the content of
>> >>> >> export-diff
>> >>> >> A@snap1 is stabled, which is almost zero.
>> >>> >> I don't think it is a designed behavior. export-diff A@snap1 should
>> >>> >> always
>> >>> >> get a stable output no matter image A is cloned or not.
>> >>> >>
>> >>> >> Please correct me if anything wrong.
>> >>> >>
>> >>> >> Thanks,
>> >>> >> Zhongyan
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >> On Tue, Nov 22, 2016 at 10:31 PM, Jason Dillaman
>> >>> >> <jdillama@xxxxxxxxxx>
>> >>> >> wrote:
>> >>> >>>
>> >>> >>> On Tue, Nov 22, 2016 at 5:31 AM, Zhongyan Gu
>> >>> >>> <zhongyan.gu@xxxxxxxxx>
>> >>> >>> wrote:
>> >>> >>> > So if initial snapshot is NOT specified, then:
>> >>> >>> > rbd export-diff image@snap1 will diff all data to snap1. this
>> >>> >>> > cmd
>> >>> >>> > equals to
>> >>> >>> > :
>> >>> >>> > rbd export image@snap1. Is my understand right or not??
>> >>> >>>
>> >>> >>>
>> >>> >>> While they will both export all data associated w/ image@snap1,
>> >>> >>> the
>> >>> >>> "export" command will generate a raw, non-sparse dump of the full
>> >>> >>> image whereas "export-diff" will export only sections of the image
>> >>> >>> that contain data. The file generated from "export" can be used
>> >>> >>> with
>> >>> >>> the "import" command to create a new image, whereas the file
>> >>> >>> generated
>> >>> >>> from "export-diff" can only be used with "import-diff" against an
>> >>> >>> existing image.
>> >>> >>>
>> >>> >>> --
>> >>> >>> Jason
>> >>> >>
>> >>> >>
>> >>> >
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Jason
>> >>
>> >>
>> >
>>
>>
>>
>> --
>> Jason
>
>
Jason
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
