Thanks for clarifying, would have been nice to protect against admin mistakes. Argument "--exclusive" is not available in jewel version of rbd-nbd. So I'd better be careful with those rbd volumes :-) Matthias On Thu, Apr 06, 2017 at 11:20:10PM -0400, Jason Dillaman wrote: > It's exclusive in that only a single client can write to an image at a > time, but it's not exclusive in that it prevents other clients from > cooperatively requesting the exclusive lock when they have an > outstanding write request. This cooperative lock transition was always > a stop-gap design to handle QEMU live-migrations. The original > intention of the feature was to prevent corruption of the object map, > fast-diff, journaling, etc features that were designed to assume only > a single client is concurrently writing to an image. > > Hopefully once QEMU 2.10+ deeply integrates VM image locking, we can > connect the new hooks to the existing librbd APIs which will disable > this behavior. Note that the rbd-nbd daemon supports an optional > "--exclusive" argument which disables the cooperative lock exchange > and adding this support to krbd is also in-progress. > > On Thu, Apr 6, 2017 at 6:16 PM, Matthias Ferdinand <mf+ml.ceph@xxxxxxxxx> wrote: > > Hello, > > > > I am building my first cluster on Ubuntu 16.04 with jewel 10.2.6, to > > host rbd images for qemu (also on Ubuntu 16.04). I've been lurking on > > this list for some time, thanks to all you regular posters for so many > > valuable insights! > > > > Tried to test the exclusive-lock rbd feature. It does not seem to > > prevent concurrent access (FS corruption) to the same rbd volume. Do I > > misinterpret its name? Do I need to configure some virsh parameters? > > > > http://lists.ceph.com/pipermail/ceph-users-ceph.com/2015-September/004857.html > > hints at live migration use case, where concurrent write access is a > > requirement. We might want to use that later, but for the moment we > > would prefer the FS to be protected against concurrent writes. > > > > root@ceph00:~# rbd info t3-x > > rbd image 't3-x': > > size 16384 MB in 4096 objects > > order 22 (4096 kB objects) > > block_name_prefix: rbd_data.3218d238e1f29 > > format: 2 > > features: layering, exclusive-lock, object-map, fast-diff, deep-flatten > > flags: > > > > After starting the VM on the first host, a lock is registered: > > > > root@ceph00:~# rbd lock ls t3-x > > There is 1 exclusive lock on this image. > > Locker ID Address > > client.205200 auto 139973185508448 192.168.xx.240:0/4012132689 > > > > Starting the same VM on another host while the first one is running, the > > registered rbd lock above stays in place, but still the second client > > can happily write to the same rbd and ruin the FS (no actual harm, just > > testing). > > > > -- > > Matthias Ferdinand > > _______________________________________________ > > ceph-users mailing list > > ceph-users@xxxxxxxxxxxxxx > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > -- > Jason _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |