Re: RGW objects has same marker and bucket id in different buckets.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Morphin,

On Thu, Apr 22, 2021 at 3:40 AM by morphin <morphinwithyou@xxxxxxxxx> wrote:
>
> Thanks for the answer.
>
> I have 2 question:
> 1- If I use different user and a bucket what will happen?  Is this
> design only for same user or user independent?

It's user independent.

> 2- If I delete the Source bucket with radosgw-admin or via S3 delete,
> what will happen to these objects?
>

The refcount on each shared object will be decreased by 1.  If there
are no more references, the objects will be garbage collected.

>
>
> Also I have more questions if you have time :)

I don't have the expertise in multisite replication to debug this.  I
don't think the issue is related to RGW's copy-sharing.

Matt

>
> Some objects are pending state due to zone-sync error. I've removed
> the master zone and set secondary to master.
> I still see the pending objects in the bucket. (below)
>
> radosgw-admin --id radosgw.srv1 object stat --bucket=descript
> --object=2020/01/17/1b819bd9-5036-4ca4-98f7-b0308e1e3017
> {
>     "name": "2020/01/17/1b819bd9-5036-4ca4-98f7-b0308e1e3017",
>     "size": 0,
>     "tag": "",
>     "attrs": {
>         "user.rgw.manifest": "",
>         "user.rgw.olh.idtag": "ivlde1avu2l3lli6i349h62c0d79ao4u",
>         "user.rgw.olh.info": "\u0001\u0001�",
>         "user.rgw.olh.pending.00000000607d4d5be0hh3lpzjd7vzt2j":
> "\u0001\u0001\u0008",
>         "user.rgw.olh.pending.00000000607d4d5c9uhlh9sf93j8lf7l":
> "\u0001\u0001\u0008",
>         "user.rgw.olh.pending.00000000607d4d5cpip1i8z8rytcnkqf":
> "\u0001\u0001\u0008",
>         "user.rgw.olh.ver": "3"
>     }
> }
>
>
> I overwrite these objects with rclone from old zone bucket to new
> created bucket on the same user at master zone.
>
> After a while I noticed that I'm getting a warning for these objects
> in rgw client log and the overwritten objects switching back to the
> corrupted objects.
>
> 2021-04-22 10:27:55.445 7f2d85fd4700  0 WARNING: couldn't find acl
> header for object, generating default
> 2021-04-22 10:27:55.445 7f2d85fd4700  1 ====== req done
> req=0x55a441452710 op status=0 http_status=200 latency=0.0200002s
> ======
> 2021-04-22 10:27:55.445 7f2d85fd4700  1 beast: 0x55a441452710:
> 10.151.101.15 - - [2021-04-22 10:27:55.0.44549s] "GET
> /descript/2020/01/17/1b819bd9-5036-4ca4-98f7-b0308e1e3017 HTTP/1.1"
> 200 0 - "aws-sdk-java/1.11.638 Linux/3.10.0-1160.11.1.el7.x86_64
> Java_HotSpot(TM)_64-Bit_Server_VM/25.281-b09 java/1.8.0_281
> groovy/2.5.6 vendor/Oracle_Corporation" -
>
> Am I doing something wrong?
> Also "sync error trim" does not work. How can I clean these errors and
> these pending objects?
>
> ceph version 14.2.16
>
>
> Have a great day.
> Regards.
>
>
> Matt Benjamin <mbenjami@xxxxxxxxxx>, 22 Nis 2021 Per, 06:08 tarihinde
> şunu yazdı:
> >
> > Hi Morphin,
> >
> > Yes, this is by design.  When an RGW object has tail chunks and is
> > copied so as to duplicate an entire tail chunk, RGW causes the
> > coincident chunk(s) to be shared.  Tail chunks are refcounted to avoid
> > leaks.
> >
> > Matt
> >
> > On Wed, Apr 21, 2021 at 4:21 PM by morphin <morphinwithyou@xxxxxxxxx> wrote:
> > >
> > > Hello.
> > >
> > > I have a rgw s3 user and the user have 2 bucket.
> > > I tried to copy objects from old.bucket to new.bucket with rclone. (in
> > > the rgw client server)
> > > After I checked the object with "radosgw-admin --bucket=new.bucket
> > > object stat $i" and I saw old.bucket id and marker id also old bucket
> > > name in the object stats.
> > >
> > > Is rgw doing this for deduplication or is it a bug?
> > > If it's not a bug then If I delete the old bucket what will happen to
> > > these objects???
> > > _______________________________________________
> > > ceph-users mailing list -- ceph-users@xxxxxxx
> > > To unsubscribe send an email to ceph-users-leave@xxxxxxx
> > >
> >
> >
> > --
> >
> > Matt Benjamin
> > Red Hat, Inc.
> > 315 West Huron Street, Suite 140A
> > Ann Arbor, Michigan 48103
> >
> > http://www.redhat.com/en/technologies/storage
> >
> > tel.  734-821-5101
> > fax.  734-769-8938
> > cel.  734-216-5309
> >
>


-- 

Matt Benjamin
Red Hat, Inc.
315 West Huron Street, Suite 140A
Ann Arbor, Michigan 48103

http://www.redhat.com/en/technologies/storage

tel.  734-821-5101
fax.  734-769-8938
cel.  734-216-5309
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux