On 04/22/2015 03:20 PM, Florian Haas wrote: > On Wed, Apr 22, 2015 at 1:02 PM, Wido den Hollander <wido@xxxxxxxx> wrote: >> On 04/22/2015 12:07 PM, Florian Haas wrote: >>> Hi everyone, >>> >>> I don't think this has been posted to this list before, so just >>> writing it up so it ends up in the archives. >>> >>> tl;dr: Using RBD storage pools with libvirt is currently broken on >>> Ubuntu trusty (LTS), and any other platform using libvirt 1.2.2. >>> >>> In libvirt 1.2.2, the rbd_create3 function is invoked, on volume >>> creation from a pool, with the stripe_count and stripe_unit parameters >>> reversed. So if you have an rbd storage pool, and you do "virsh >>> vol-create-as" or something equivalent, then instead of a stripe count >>> of one and a stripe size of 4MB, you get 4194304 1-byte stripes. >>> Needless to say, this renders the volume excruciatingly slow to the >>> point of not being usable. Volume deletion also takes on the order of >>> minutes even for an empty volume. >>> >>> This issue was introduced in libvirt 1.2.1, and was fixed for 1.2.4, >>> but Ubuntu 14.04 LTS (which is on 1.2.2) evidently never backported >>> that fix. >>> >> >> Oops... Did I do this? I think so. I messed up with the arguments. >> >> My apologies! > > No need to apologize; after all it *is* fixed upstream, and AFAICT no > other major distros are affected. It's just that Ubuntu really need to > backport that one-line fix if they want libvirt RBD pool functionality > to work in their current LTS. > > I'm not entirely sure, though, why virStorageBackendRBDCreateImage() > enables striping unconditionally; could you explain the reasoning > behind that? > When working on this with Josh some time ago we had to come up with a way to create RBD format 2 images before the RBD default format option was available. The only way to do it was to set the stripe size and count specifically and check in libvirt which version librbd was. This code could be changed again if we want by doing: rados_conf_set(ptr->cluster, "rbd_default_format", "2"); rbd_create(.....); Any recent librbd library would create format 2 images then. Since there is no way to feed any metadata to libvirt when creating an image this is the only way to go. Wido > Cheers, > Florian > -- Wido den Hollander 42on B.V. Ceph trainer and consultant Phone: +31 (0)20 700 9902 Skype: contact42on _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |