Hi,
On 05/31/2012 09:12 AM, Alexandre DERUMIER wrote:
Hi,
I trying to delete some rbd images with rbd rm,
and it seem to be "slow" with big images.
I'm testing it with just create a new image (1TB):
# time rbd -p pool1 create --size 1000000 image2
real 0m0.031s
user 0m0.015s
sys 0m0.010s
then just delete it, without having writed nothing in image
# time rbd -p pool1 rm image2
Removing image: 100% complete...done.
real 1m45.558s
user 0m14.683s
sys 0m17.363s
same test with 100GB
# time rbd -p pool1 create --size 100000 image2
real 0m0.032s
user 0m0.016s
sys 0m0.007s
# time rbd -p pool1 rm image2
Removing image: 100% complete...done.
real 0m10.499s
user 0m1.488s
sys 0m1.720s
I'm using journal in tmpfs, 3 servers, 15 osds with 1disk 15K (xfs)
network bandwith,diskio,cpu are low.
Is it the normal behaviour ? Maybe some xfs tuning could help ?
It's in the nature of RBD.
A RBD image consists of multiple 4MB (default) RADOS objects.
Let's say you have a disk of 40GB, that will contain 10.000 4MB RADOS
objects, you can find those objects by doing: rados -p rbd ls
Now, when you create a new image only the header is writting, but no
object is written.
When you start writing to a RBD image you will be writing to one of the
4MB objects. When it doesn't exist it will be created.
So when you install your VM it will create objects, but not all of them.
RBD knows which RADOS objects to access by three parameters:
* Image name
* Image size
* Stripe size (4MB)
So when your VM access for byte Y until Z on the disk, RBD knows which
object to access by calculating this.
Now, when you start removing the image there is no way of knowing which
object exists and which doesn't, so RBD will try to remove all objects.
In the case of a fresh image this results in 10.000 RADOS remove
operations for non-existent objects and that is slow.
Wido
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
