I am developing a python application (using rbd.py) that requires querying information about tens of thousands of rbd images. I have noticed that the number of threads in my process grow linearly with each Image object that is created.
After creating about 800 Image objects (that all share a single ioctx), my process already has more than 2000 threads. I get the thread count using `ps huH p <pid> | wc -l`.
If I call close() on each Image object after operating on it then the threads are cleaned up. However, I want to cache these objects and reuse them since it is expensive to create tens of thousands of these objects all the time.
Is it correct for librbd to create 4-5 threads for each Image object created?
For example, I'm doing something similar to:
-------------
import rbd
import rados
cluster = rados.Rados(conffile='my_ceph.conf')
cluster.connect()
ioctx = cluster.open_ioctx('mypool')
# With each object, new threads are created until close() is called
image0 = rbd.Image(ioctx, 'myimage0')
image1 = rbd.Image(ioctx, 'myimage1')
...
image9000 = rbd.Image(ioctx, 'myimage9000')
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
