does the RBD client block write when the Watcher times out?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hello.

I'm currently verifying the behavior of RBD on failure. I'm wondering
about the consistency of RBD images after network failures. As a
result of my investigation, I found that RBD sets a Watcher to RBD
image if a client mounts this volume to prevent multiple mounts. In
addition, I found that if the client is isolated from the network for
a long time, the Watcher is released. However, the client still mounts
this image. In this situation, if another client can also mount this
image and the image is writable from both clients, data corruption
occurs. Could you tell me whether this is a realistic scenario?

I tested the following case on which the watcher was released by hand
and detected data corruption.

1. Release the Watcher on the node (A) that mounts the RBD using the
`ceph osd blocklist add` command
2. Another node (B) mounts the RBD volume.
3. Unblock using the `ceph osd blocklist rm` command
4. Write from node (B) (write successfully)
5. Write from node (A) (can be written successfully from the
application's point of view. In fact, the write fails)
6. Write content at node (A) is lost.

In this case, I released the watcher by hand to emulate the timeout
due to network failure. It's because I couldn't emulate real network
failure in this test.

I considered using exclusive lock and restricting access to those from
a single node. However, we gave up on that as blocking writes entirely
would make snapshots non-functional.

The version of Ceph we are using is v17.2.6.

Best regards,
Yuma.
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux