Hello, On Mon, 26 Mar 2018 18:20:22 -0300 Thiago Gonzaga wrote: > Hey Guys, > > Is it normal to get all process stuck while a image is deleting? > If you'd tell us all the details of your cluster (SW and HW) we might be more helpful and point to specific issues. So to generically answer your question, yes. Deleting images, snapshots and a number of what would be considered normal standard tasks for Ceph can create insane I/O storms, completely saturating your storage. Which you would see if you monitored your storage nodes and can quickly verify with atop or iostat. Some of these things can be mitigated by throttling Ceph down where such knobs exist, others have gotten better over time with newer versions, but penultimately your storage system will be the bottleneck. Christian > I'm removing 21g layering image and it takes ages, while the image is > removing I have lots of problems in the log like this > > I ran rbd create rbd/test -s 21G --image-feature layering > deleted the image while deleted that tried to create another one and format > it > root@ceph-mon-3:/home/tgonzaga# rbd create rbd/test2 -s 10G --image-feature > layering > root@ceph-mon-3:/home/tgonzaga# rbd map test2 > /dev/rbd0 > > it's stuck until the image gets completely deleted > > 2018-03-26 21:15:30.431835 [WRN] Health check update: 551 slow requests > are blocked > 32 sec (REQUEST_SLOW) > 2018-03-26 21:15:00.373288 [WRN] Health check update: 555 slow requests > are blocked > 32 sec (REQUEST_SLOW) > 2018-03-26 21:14:54.365829 [WRN] Health check update: 533 slow requests > are blocked > 32 sec (REQUEST_SLOW) > 2018-03-26 21:14:44.351739 [WRN] Health check update: 531 slow requests > are blocked > 32 sec (REQUEST_SLOW) > 2018-03-26 21:14:38.342610 [WRN] Health check update: 541 slow requests > are blocked > 32 sec (REQUEST_SLOW) > 2018-03-26 21:14:10.734288 [WRN] Health check update: 555 slow requests > are blocked > 32 sec (REQUEST_SLOW) > 2018-03-26 21:14:05.733855 [WRN] Health check update: 557 slow requests > are blocked > 32 sec (REQUEST_SLOW) > 2018-03-26 21:14:00.288994 [WRN] Health check update: 538 slow requests > are blocked > 32 sec (REQUEST_SLOW) > 2018-03-26 21:13:50.275008 [WRN] Health check update: 534 slow requests > are blocked > 32 sec (REQUEST_SLOW) > 2018-03-26 21:13:44.267499 [WRN] Health check update: 536 slow requests > are blocked > 32 sec (REQUEST_SLOW) > 2018-03-26 21:13:38.258227 [WRN] Health check update: 171 slow requests > are blocked > 32 sec (REQUEST_SLOW) > 2018-03-26 21:13:32.250539 [WRN] Health check update: 170 slow requests > are blocked > 32 sec (REQUEST_SLOW) > 2018-03-26 21:13:26.241810 [WRN] Health check update: 163 slow requests > are blocked > 32 sec (REQUEST_SLOW) > > And I can't perform anything on other images. > > Any ideas? > > Regards > > > *Thiago Gonzaga* > SaaSOps Software Architect > o. 1 (512) 2018-287 x2119 > Skype: thiago.gonzaga20 > [image: Aurea] > <http://www.aurea.com/?utm_source=email-signature&utm_medium=email> -- Christian Balzer Network/Systems Engineer chibi@xxxxxxx Rakuten Communications _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |