The 'rbd_concurrent_management_ops' setting controls how many concurrent, in-flight RADOS object delete operations are possible per image removal. The default is only 10, so given ten 10 images being deleted concurrently, I am actually surprised that blocked all IO from your VMs. Adding support for limiting the maximum number of concurrent image deletions would definitely be an OpenStack enhancement. There is an open blueprint for optionally utilizing the RBD trash instead of having Cinder delete the images [1], which would allow you to defer deletions to whenever is convenient. Additionally, once Ceph adds support for backend QoS (fingers crossed in Nautilus), we can change librbd to flag all IO for maintenance activities to background (best effort) priority, which might be the best long-term solution. [1] https://blueprints.launchpad.net/cinder/+spec/rbd-deferred-volume-deletion On Wed, Jun 6, 2018 at 6:40 AM, Yao Guotao <yaoguo_tao@xxxxxxx> wrote: > Hi Cephers, > > We use Ceph with Openstack by librbd library. > > Last week, my colleague delete 10 volumes from Openstack dashboard at the > same time, each volume has about 1T used. > During this time, the disk of OSDs are busy, and there have no I/O for > normal vm. > > So, I want to konw if there are any parameters that can be set to throttle? > > I find a parameter about rbd op is 'rbd_concurrent_management_ops'. > I am trying to figure out how it works in code, and I find the parameter can > only control the asyncchronous deletion of all objects of an image. > > Besides, Should it be controlled at Openstack Nova or Cinder layer? > > Thanks, > Yao Guotao > > > > > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- Jason _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |