Hi Amit: Both clusters have a lot of recovering shards. Actually I do not know if it’s normal or not. 🙁 The rgw_rados_hander is the default value, I have not touched this parameter. Do I need to increase this value? Thanks Amit Ghadge <amitg.b14@xxxxxxxxx>于2021年4月26日 周一下午10:42写道： > Both clusters show sync status are up to date or is there any behind > shards, how many gw endpoints you configure and what rgw_rados_hander > parameters are set? > > On Mon, Apr 26, 2021 at 8:59 AM 特木勒 <twl007@xxxxxxxxx> wrote: > >> Another problem I notice for a new bucket, the first object in the bucket >> will not be sync. the sync will start with the second object. I tried to >> fix the index on the bucket and manually rerun bucket sync, but the first >> object still does not sync with secondary cluster. >> >> Do you have any ideas for this issue? >> >> Thanks >> >> 特木勒 <twl007@xxxxxxxxx> 于2021年4月26日周一 上午11:16写道： >> >>> Hi Istvan: >>> >>> Thanks Amit's suggestion. >>> >>> I followed his suggestion to fix bucket index and re-do sync on buckets, >>> but it still did not work for me. >>> >>> Then I tried to use bucket rewrite command to rewrite all the objects in >>> buckets and it works for me. I think the reason is there's something wrong >>> with bucket index and rewrite has rebuilt the index. >>> >>> Here's the command I use: >>> `sudo radosgw-admin bucket rewrite -b BUCKET-NAME --min-rewrite-size 0` >>> >>> Maybe you can try this to fix the sync issues. >>> >>> @Amit Ghadge <amitg.b14@xxxxxxxxx> Thanks for your suggestions. Without >>> your suggestions, I will not notice something wrong with index part. >>> >>> Thanks :) >>> >>> Szabo, Istvan (Agoda) <Istvan.Szabo@xxxxxxxxx> 于2021年4月26日周一 上午9:57写道： >>> >>>> Hi, >>>> >>>> >>>> >>>> No, doesn’t work, now we will write our own sync app for ceph, I gave >>>> up. >>>> >>>> >>>> >>>> Istvan Szabo >>>> Senior Infrastructure Engineer >>>> --------------------------------------------------- >>>> Agoda Services Co., Ltd. >>>> e: istvan.szabo@xxxxxxxxx >>>> --------------------------------------------------- >>>> >>>> >>>> >>>> *From:* 特木勒 <twl007@xxxxxxxxx> >>>> *Sent:* Friday, April 23, 2021 7:50 PM >>>> *To:* Szabo, Istvan (Agoda) <Istvan.Szabo@xxxxxxxxx> >>>> *Cc:* ceph-users@xxxxxxx >>>> *Subject:* Re: [Suspicious newsletter] RGW: Multiple Site >>>> does not sync olds data >>>> >>>> >>>> >>>> Hi Istvan: >>>> >>>> >>>> >>>> We just upgraded whole cluster to 15.2.10 and the multiple site still >>>> cannot sync whole objects to secondary cluster. 🙁 >>>> >>>> >>>> >>>> Do you have any suggestions on this? And I open another issues in ceph >>>> tracker site: >>>> >>>> https://tracker.ceph.com/issues/50474 >>>> >>>> >>>> >>>> Hope someone could go to check this issue. >>>> >>>> >>>> >>>> Thanks >>>> >>>> >>>> >>>> 特木勒 <twl007@xxxxxxxxx>于2021年3月22日 周一下午9:08写道： >>>> >>>> Thank you~ >>>> >>>> >>>> >>>> I will try to upgrade cluster too. Seem like this is the only way for >>>> now. 😭 >>>> >>>> >>>> >>>> I will let you know once I complete testing. :) >>>> >>>> >>>> >>>> Have a good day >>>> >>>> >>>> >>>> Szabo, Istvan (Agoda) <Istvan.Szabo@xxxxxxxxx>于2021年3月22日 周一下午3:38写道： >>>> >>>> Yeah, doesn't work. Last week they fixed my problem ticket which caused >>>> the crashes, and due to the crashes stopped the replication I'll give a try >>>> this week again after the update if the daemon doesn't crash, maybe it will >>>> work, because if crash hasn't happened, the data was synced. Fingers >>>> crossed ;) Don't give up 😄 >>>> ------------------------------ >>>> >>>> *From:* 特木勒 <twl007@xxxxxxxxx> >>>> *Sent:* Monday, March 22, 2021 1:38 PM >>>> *To:* Szabo, Istvan (Agoda) <Istvan.Szabo@xxxxxxxxx> >>>> *Cc:* ceph-users@xxxxxxx <ceph-users@xxxxxxx> >>>> >>>> >>>> *Subject:* Re: [Suspicious newsletter] RGW: Multiple Site >>>> does not sync olds data >>>> >>>> >>>> >>>> Hi Istvan: >>>> >>>> >>>> >>>> Do you have any update on directional sync? >>>> >>>> >>>> >>>> I am trying to upgrade cluster to 15.2.10 to see if the problem is >>>> solved. :( >>>> >>>> >>>> >>>> Thanks >>>> >>>> >>>> >>>> Szabo, Istvan (Agoda) <Istvan.Szabo@xxxxxxxxx> 于2021年3月1日周一 上午10:01写道： >>>> >>>> So-so. I had some interruption so it failed on one site, but the other >>>> is kind of working. This is the first time when I saw data caught up in the >>>> radosgw-admin data sync status on 1 side. >>>> >>>> Today will finish the other problematic site, I’ll let you know the >>>> result is it working or not. >>>> >>>> >>>> >>>> Istvan Szabo >>>> Senior Infrastructure Engineer >>>> --------------------------------------------------- >>>> Agoda Services Co., Ltd. >>>> e: istvan.szabo@xxxxxxxxx >>>> --------------------------------------------------- >>>> >>>> >>>> >>>> *From:* 特木勒 <twl007@xxxxxxxxx> >>>> *Sent:* Sunday, February 28, 2021 1:34 PM >>>> *To:* Szabo, Istvan (Agoda) <Istvan.Szabo@xxxxxxxxx> >>>> *Cc:* ceph-users@xxxxxxx >>>> *Subject:* Re: [Suspicious newsletter] RGW: Multiple Site >>>> does not sync olds data >>>> >>>> >>>> >>>> Email received from outside the company. If in doubt don't click links >>>> nor open attachments! >>>> ------------------------------ >>>> >>>> Hi Istvan: >>>> >>>> >>>> >>>> Thanks for your reply. >>>> >>>> >>>> >>>> Does directional sync solve the problem? I tried to run `radosgw-admin >>>> sync init`, bit it still did not work. :( >>>> >>>> >>>> >>>> Thanks >>>> >>>> >>>> >>>> Szabo, Istvan (Agoda) <Istvan.Szabo@xxxxxxxxx> 于2021年2月26日周五 上午7:47写道： >>>> >>>> Same for me, 15.2.8 also. >>>> I’m trying directional sync now, looks like symmetrical has issue. >>>> >>>> Istvan Szabo >>>> Senior Infrastructure Engineer >>>> --------------------------------------------------- >>>> Agoda Services Co., Ltd. >>>> e: istvan.szabo@xxxxxxxxx<mailto:istvan.szabo@xxxxxxxxx> >>>> --------------------------------------------------- >>>> >>>> On 2021. Feb 26., at 1:03, 特木勒 <twl007@xxxxxxxxx> wrote: >>>> >>>> Email received from outside the company. If in doubt don't click links >>>> nor open attachments! >>>> ________________________________ >>>> >>>> Hi all: >>>> >>>> ceph version: 15.2.7 (88e41c6c49beb18add4fdb6b4326ca466d931db8) >>>> >>>> I have a strange question, I just create a multiple site for Ceph >>>> cluster. >>>> But I notice the old data of source cluster is not synced. Only new data >>>> will be synced into second zone cluster. >>>> >>>> Is there anything I need to do to enable full sync for bucket or this >>>> is a >>>> bug? >>>> >>>> Thanks >>>> _______________________________________________ >>>> ceph-users mailing list -- ceph-users@xxxxxxx >>>> To unsubscribe send an email to ceph-users-leave@xxxxxxx >>>> >>>> ________________________________ >>>> This message is confidential and is for the sole use of the intended >>>> recipient(s). It may also be privileged or otherwise protected by copyright >>>> or other legal rules. If you have received it by mistake please let us know >>>> by reply email and delete it from your system. It is prohibited to copy >>>> this message or disclose its content to anyone. Any confidentiality or >>>> privilege is not waived or lost by any mistaken delivery or unauthorized >>>> disclosure of the message. All messages sent to and from Agoda may be >>>> monitored to ensure compliance with company policies, to protect the >>>> company's interests and to remove potential malware. Electronic messages >>>> may be intercepted, amended, lost or deleted, or contain viruses. >>>> >>>> >>>> >>>> Internal >>>> >>>> _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |