Hi, I had a similar problem while resharding an oversized non-sharded bucket in Jewel (10.2.7), the bi_list exited with ERROR: bi_list(): (4) Interrupted system call at, what seemed like the very end of the operation. I went ahead and resharded the bucket anyway and the reshard process ended the same way, seemingly at the end. Reshard didn't link the bucket to new instance id though so I had to do that by hand and then purge the index from the old instance id. Note that I didn't care about the data in the bucket, I just wanted to reshard the index so I could delete the bucket without my radosgw and osds crashing due to out of memory issues. Regards, Andreas On 4 July 2017 at 20:46, Maarten De Quick <mdequick85@xxxxxxxxx> wrote: > Hi, > > Background: We're having issues with our index pool (slow requests / time > outs causes crashing of an OSD and a recovery -> application issues). We > know we have very big buckets (eg. bucket of 77 million objects with only 16 > shards) that need a reshard so we were looking at the resharding process. > > First thing we would like to do is making a backup of the bucket index, but > this failed with: > > # radosgw-admin -n client.radosgw.be-west-3 bi list > --bucket=priv-prod-up-alex > /var/backup/priv-prod-up-alex.list.backup > 2017-07-03 21:28:30.325613 7f07fb8bc9c0 0 System already converted > ERROR: bi_list(): (4) Interrupted system call > > When I grep for "idx" and I count these: > # grep idx priv-prod-up-alex.list.backup | wc -l > 2294942 > When I do a bucket stats for that bucket I get: > # radosgw-admin -n client.radosgw.be-west-3 bucket stats > --bucket=priv-prod-up-alex | grep num_objects > 2017-07-03 21:33:05.776499 7faca49b89c0 0 System already converted > "num_objects": 20148575 > > It looks like there are 18 million objects missing and the backup is not > complete (not sure if that's a correct assumption?). We're also afraid that > the resharding command will face the same issue. > Has anyone seen this behaviour before or any thoughts on how to fix it? > > We were also wondering if we really need the backup. As the resharding > process creates a complete new index and keeps the old bucket, is there > maybe a possibility to relink your bucket to the old bucket in case of > issues? Or am I missing something important here? > > Any help would be greatly appreciated, thanks! > > Regards, > Maarten > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |