Bucket deletion is somewhat slow, several days to delete a large bucket is not unusual. BTW: you can speed it up with --max-concurrent-ios=XXX (default is 32, documentation in Nautilus for the switch is wrong, it does work for deleteions, docs are fixed in master) Paul -- Paul Emmerich Looking for help with your Ceph cluster? Contact us at https://croit.io croit GmbH Freseniusstr. 31h 81247 München www.croit.io Tel: +49 89 1896585 90 On Tue, Oct 29, 2019 at 7:38 PM EDH - Manuel Rios Fernandez <mriosfer@xxxxxxxxxxxxxxxx> wrote: > > As update, > > > > It continues.. > > > > 2019-10-29 19:36:48.787 7fc5ae22c700 0 abort_bucket_multiparts WARNING : aborted 2437000 incomplete multipart uploads > > > > How to get debug for upload ? > > > > Regards > > > > > > De: EDH - Manuel Rios Fernandez <mriosfer@xxxxxxxxxxxxxxxx> > Enviado el: lunes, 28 de octubre de 2019 14:18 > Para: ceph-users@xxxxxxx > Asunto: After delete 8.5M Objects in a bucket still 500K left > > > > Hi Ceph’s! > > > > We started deteling a bucket several days ago. Total size 47TB / 8.5M objects. > > > > Now we see the cli bucket rm stucked and by console drop this messages. > > > > [root@ceph-rgw03 ~]# 2019-10-28 13:55:43.880 7f0dd92c9700 0 abort_bucket_multiparts WARNING : aborted 1000 incomplete multipart uploads > > 2019-10-28 13:56:24.021 7f0dd92c9700 0 abort_bucket_multiparts WARNING : aborted 2000 incomplete multipart uploads > > 2019-10-28 13:57:04.726 7f0dd92c9700 0 abort_bucket_multiparts WARNING : aborted 3000 incomplete multipart uploads > > 2019-10-28 13:57:45.424 7f0dd92c9700 0 abort_bucket_multiparts WARNING : aborted 4000 incomplete multipart uploads > > 2019-10-28 13:58:25.905 7f0dd92c9700 0 abort_bucket_multiparts WARNING : aborted 5000 incomplete multipart uploads > > 2019-10-28 13:59:06.898 7f0dd92c9700 0 abort_bucket_multiparts WARNING : aborted 6000 incomplete multipart uploads > > 2019-10-28 13:59:47.829 7f0dd92c9700 0 abort_bucket_multiparts WARNING : aborted 7000 incomplete multipart uploads > > 2019-10-28 14:00:42.102 7f0dd92c9700 0 abort_bucket_multiparts WARNING : aborted 8000 incomplete multipart uploads > > 2019-10-28 14:01:23.829 7f0dd92c9700 0 abort_bucket_multiparts WARNING : aborted 9000 incomplete multipart uploads > > 2019-10-28 14:02:06.028 7f0dd92c9700 0 abort_bucket_multiparts WARNING : aborted 10000 incomplete multipart uploads > > 2019-10-28 14:02:48.648 7f0dd92c9700 0 abort_bucket_multiparts WARNING : aborted 11000 incomplete multipart uploads > > 2019-10-28 14:03:29.807 7f0dd92c9700 0 abort_bucket_multiparts WARNING : aborted 12000 incomplete multipart uploads > > 2019-10-28 14:04:11.180 7f0dd92c9700 0 abort_bucket_multiparts WARNING : aborted 13000 incomplete multipart uploads > > 2019-10-28 14:04:52.396 7f0dd92c9700 0 abort_bucket_multiparts WARNING : aborted 14000 incomplete multipart uploads > > 2019-10-28 14:05:33.050 7f0dd92c9700 0 abort_bucket_multiparts WARNING : aborted 15000 incomplete multipart uploads > > 2019-10-28 14:06:13.652 7f0dd92c9700 0 abort_bucket_multiparts WARNING : aborted 16000 incomplete multipart uploads > > 2019-10-28 14:06:54.806 7f0dd92c9700 0 abort_bucket_multiparts WARNING : aborted 17000 incomplete multipart uploads > > 2019-10-28 14:07:35.867 7f0dd92c9700 0 abort_bucket_multiparts WARNING : aborted 18000 incomplete multipart uploads > > 2019-10-28 14:08:16.886 7f0dd92c9700 0 abort_bucket_multiparts WARNING : aborted 19000 incomplete multipart uploads > > 2019-10-28 14:08:57.711 7f0dd92c9700 0 abort_bucket_multiparts WARNING : aborted 20000 incomplete multipart uploads > > 2019-10-28 14:09:38.032 7f0dd92c9700 0 abort_bucket_multiparts WARNING : aborted 21000 incomplete multipart uploads > > 2019-10-28 14:10:18.377 7f0dd92c9700 0 abort_bucket_multiparts WARNING : aborted 22000 incomplete multipart uploads > > 2019-10-28 14:10:58.833 7f0dd92c9700 0 abort_bucket_multiparts WARNING : aborted 23000 incomplete multipart uploads > > 2019-10-28 14:11:39.078 7f0dd92c9700 0 abort_bucket_multiparts WARNING : aborted 24000 incomplete multipart uploads > > 2019-10-28 14:12:24.731 7f0dd92c9700 0 abort_bucket_multiparts WARNING : aborted 25000 incomplete multipart uploads > > 2019-10-28 14:13:12.176 7f0dd92c9700 0 abort_bucket_multiparts WARNING : aborted 26000 incomplete multipart uploads > > > > > > Bucket stats show 500K objects left. Looks like bucket rm is trying to abort all incompleted mutipart. But in bucket stats this operation is not reflected removing objects from stats. > > > > May be wait to get up 500K or it’s a bug? > > > > Regards > > Manuel > > > > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |