On 11:10 Wed 22 Jul, Ilya Dryomov wrote: > On Wed, Jul 22, 2020 at 7:46 AM Liu, Changcheng > <changcheng.liu@xxxxxxxxx> wrote: > > > > Hi all, > > I've checked below document: > > https://docs.ceph.com/docs/master/rbd/rbd-exclusive-locks/ > > > > The content is contradictory to the result shown in below experiment: > > The experient shows that another process could still write data to rbd > > volume while there already has a process write to the same rbd volume > > continuously. > > This is the expected behaviour. Exclusive lock is a cooperative > mechanism that ensures that only a single client is able to write > to the image and update its metadata (such as the object map) > at any given moment, not until the client exits. It is acquired > automatically and the ownership is transparently transitioned between > clients. In your example, "second" wakes up and requests the lock > from "first", "first" releases it, "second" performs its write, > "first" reacquires the lock and goes on. > > If you want to disable transparent lock transitions, you need to > acquire the lock manually with RBD_LOCK_MODE_EXCLUSIVE: @Ilya: Thanks for your info. The transparent lock transition could be disabled by acquring the lock with RBD_LOCK_MODE_EXCLUSIVE. After "first" process acquire the lock with "RBD_LOCK_MODE_EXCLUSIVE", is it possible for another process to be notified that the lock is released whatever the "first" process exit gracefully or be killed? I write below program to run "another process". If I manully remove the lock, "another process" could be notified. However, if the "first" process exit gracefully or be killed, "another process"'s handle_notify won't be called at all. 1 #include <rbd/librbd.hpp> 2 #include <rados/librados.hpp> 3 4 #include <cstring> 5 #include <iostream> 6 #include <string> 7 8 class TestWatcher { 9 public: 10 librados::Rados rados; 11 librbd::RBD rbd; 12 librbd::Image image; 13 librados::IoCtx io_ctx; 14 15 std::string pool_name; 16 std::string image_name; 17 18 TestWatcher(std::string pool_name = "rbd", 19 std::string image_name = "fio_test") 20 : pool_name(pool_name), image_name(image_name) { 21 int ret = rados.init("admin"); 22 if (ret < 0) { 23 std::cout << "failed to initialize rados" << std::endl; 24 exit(1); 25 } 26 27 ret = rados.conf_read_file("ceph.conf"); 28 if (ret < 0) { 29 std::cout << "failed to parse ceph.conf" << std::endl; 30 exit(1); 31 } 32 33 ret = rados.connect(); 34 if (ret < 0) { 35 std::cout << "failed to connect to rados cluster" << std::endl; 36 exit(1); 37 } 38 39 ret = rados.ioctx_create(pool_name.c_str(), io_ctx); 40 if (ret < 0) { 41 rados.shutdown(); 42 std::cout << "failed to create ioctx" << std::endl; 43 exit(1); 44 } 45 46 ret = rbd.open(io_ctx, image, image_name.c_str()); 47 if (ret < 0) { 48 io_ctx.close(); 49 rados.shutdown(); 50 std::cout << "failed to open rbd image" << std::endl; 51 exit(1); 52 } else { 53 std::cout << "open image succeeded" << std::endl; 54 } 55 } 56 57 ~TestWatcher() { 58 image.close(); 59 io_ctx.close(); 60 rados.shutdown(); 61 62 if (watch_ctx != nullptr) { 63 delete watch_ctx; 64 watch_ctx = nullptr; 65 } 66 } 67 68 class WatchCtx: public librbd::UpdateWatchCtx { 69 private: 70 TestWatcher &_test_watcher; 71 public: 72 explicit WatchCtx(TestWatcher &test_watcher): 73 _test_watcher(test_watcher) { 74 } 75 76 int list_watchers() { 77 std::list<librbd::image_watcher_t> watcher_list; 78 int r = _test_watcher.image.list_watchers(watcher_list); 79 if (r >= 0) { 80 for (auto it = watcher_list.cbegin(); it != watcher_list.cend(); ++it) { 81 std::cout << "addr: " << it->addr.c_str() << ", " 82 << "id: " << it->id << ", " 83 << "cookie: " << it->cookie << std::endl; 84 } 85 } 86 return r; 87 } 88 89 int list_lockers() { 90 std::list<librbd::locker_t> lockers; 91 std::string tag; 92 bool exclusive; 93 int r = _test_watcher.image.list_lockers(&lockers, &exclusive, &tag); 94 if (r >= 0) { 95 for (auto it = lockers.cbegin(); it != lockers.cend(); ++it) { 96 std::cout << "client: " << it->client.c_str() << ", " 97 << "cookie: " << it->cookie.c_str() << ", " 98 << "address: " << it->address.c_str() << std::endl; 99 } 100 } 101 return r; 102 } 103 void handle_notify() override { 104 std::cout << "event comming" << std::endl; 105 } 106 }; 107 108 int list_watchers() { 109 watch_ctx = new WatchCtx(*this); 110 int r = watch_ctx->list_watchers(); 111 delete watch_ctx; 112 watch_ctx = nullptr; 113 return r; 114 } 115 116 int list_lockers() { 117 watch_ctx = new WatchCtx(*this); 118 int r = watch_ctx->list_lockers(); 119 delete watch_ctx; 120 watch_ctx = nullptr; 121 return r; 122 } 123 124 125 int update_watch(uint64_t *phandle) { 126 watch_ctx = new WatchCtx(*this); 127 return image.update_watch(watch_ctx, phandle); 128 } 129 WatchCtx* watch_ctx = nullptr; 130 }; 131 132 int main(void) { 133 TestWatcher testwatcher; 134 testwatcher.list_watchers(); 135 testwatcher.list_lockers(); 136 uint64_t handle = 0; 137 testwatcher.update_watch(&handle); 138 while(1); 139 } B.R. Changcheng > > > Thanks, > > Ilya _______________________________________________ Dev mailing list -- dev@xxxxxxx To unsubscribe send an email to dev-leave@xxxxxxx
