What if I'm holding this lock and somebody else is trying to reacquire the lock.
How do I get notified about it?
On Fri, Aug 19, 2016 at 5:48 AM, Mykola Golub <mgolub@xxxxxxxxxxxx> wrote:
> On Thu, Aug 18, 2016 at 09:20:02PM -0700, Victor Denisov wrote:
>> Could you please point me to the place in source code where writer
>> acquires an exclusive lock on the image.
>
> Grep for 'exclusive_lock->request_lock'. Basically, what you need
> (after opening the image) is:
>
> ```
> C_SaferCond lock_ctx;
> {
> RWLock::WLocker l(ictx->owner_lock);
>
> if (ictx->exclusive_lock == nullptr) {
> // exclusive-lock feature is not enabled
> return -EINVAL;
> }
>
> // Request the lock. If it is currently owned by another client,
> // RPC message will be sent to the client to release the lock.
> ictx->exclusive_lock->request_lock(&lock_ctx);
> } // release owner_lock before waiting to avoid potential deadlock
>
> int r = lock_ctx.wait();
> if (r < 0) {
> return r;
> }
>
> RWLock::RLocker l(ictx->owner_lock);
> if (ictx->exclusive_lock == nullptr || !ictx->exclusive_lock->is_lock_owner()) {
> // failed to acquire exclusive lock
> return -EROFS;
> }
>
> // At this point lock is acquired
> ...
>
> ```
>
> You might want to look at this PR
>
> https://github.com/ceph/ceph/pull/9592
>
> where we discuss adding API methods to directly acquire and release
> the exclusive lock. You don't need the API, but will find examples in
> the patch, and also useful comments from Jason.
>
> --
> Mykola Golub
>
>> I presume you were talking about the feature:
>> exclusive_lock, shared_lock which can be used from command line using
>> commands lock list, lock break.
>>
>> On Thu, Aug 18, 2016 at 5:47 PM, Jason Dillaman <jdillama@xxxxxxxxxx> wrote:
>> > There is already a "request lock" RPC message and this is already handled
>> > transparently within librbd when you attempt to acquire the lock and another
>> > client owns it.
>> >
>> >
>> > On Thursday, August 18, 2016, Victor Denisov <vdenisov@xxxxxxxxxxxx> wrote:
>> >>
>> >> If an image already has a writer who owns the lock,
>> >> should I implement a notification that allows to ask the writer to
>> >> release the lock,
>> >> is there already a standard way to intercept the exclusive lock?
>> >>
>> >> On Tue, Aug 16, 2016 at 6:29 AM, Jason Dillaman <jdillama@xxxxxxxxxx>
>> >> wrote:
>> >> > ... one more thing:
>> >> >
>> >> > I was also thinking that we need a new RBD feature bit to be used to
>> >> > indicate that an image is part of a consistency group to prevent older
>> >> > librbd clients from removing the image or group snapshots. This could
>> >> > be a RBD_FEATURES_RW_INCOMPATIBLE feature bit so older clients can
>> >> > still open the image R/O while its part of a group.
>> >> >
>> >> > On Tue, Aug 16, 2016 at 9:26 AM, Jason Dillaman <jdillama@xxxxxxxxxx>
>> >> > wrote:
>> >> >> Way back in April when we had the CDM, I was originally thinking we
>> >> >> should implement option 3. Essentially, you have a prepare group
>> >> >> snapshot RPC message that extends a "paused IO" lease to the caller.
>> >> >> When that lease expires, IO would automatically be resumed even if the
>> >> >> group snapshot hasn't been created yet. This would also require
>> >> >> commit/abort group snapshot RPC messages.
>> >> >>
>> >> >> However, thinking about this last night, here is another potential
>> >> >> option:
>> >> >>
>> >> >> Option 4 - require images to have the exclusive lock feature before
>> >> >> they can be added to a consistency group (and prevent disabling of
>> >> >> exclusive-lock while they are part of a group). Then librbd, via the
>> >> >> rbd CLI (or client application of the rbd consistency group snap
>> >> >> create API), can co-operatively acquire the lock from all active image
>> >> >> clients within the group (i.e. all IO has been flushed and paused) and
>> >> >> can proceed with snapshot creation. If the rbd CLI dies, the normal
>> >> >> exclusive lock handling process will automatically take care of
>> >> >> re-acquiring the lock from the dead client and resuming IO.
>> >> >>
>> >> >> This option not only re-uses existing code, it would also eliminate
>> >> >> the need to add/update the RPC messages for prepare/commit/abort
>> >> >> snapshot creation to support group snapshots (since it could all be
>> >> >> handled internally).
>> >> >>
>> >> >> On Mon, Aug 15, 2016 at 7:46 PM, Victor Denisov <vdenisov@xxxxxxxxxxxx>
>> >> >> wrote:
>> >> >>> Gentlemen,
>> >> >>>
>> >> >>> I'm writing to you to ask for your opinion regarding quiescing writes.
>> >> >>>
>> >> >>> Here is the situation. In order to take snapshots of all images in a
>> >> >>> consistency group,
>> >> >>> we first need to quiesce all the image writers in the consistency
>> >> >>> group.
>> >> >>> Let me call
>> >> >>> group client - a client which requests a consistency group to take a
>> >> >>> snapshot.
>> >> >>> Image client - the client that writes to an image.
>> >> >>> Let's say group client starts sending notify_quiesce to all image
>> >> >>> clients that write to the images in the group. After quiescing half of
>> >> >>> the image clients the group client can die.
>> >> >>>
>> >> >>> It presents us with a dilemma - what should we do with those quiesced
>> >> >>> image clients.
>> >> >>>
>> >> >>> Option 1 - is to wait till someone manually runs recover for that
>> >> >>> consistency group.
>> >> >>> We can show warning next to those unfinished groups when user runs
>> >> >>> group list command.
>> >> >>> There will be a command like group recover, which allows users to
>> >> >>> rollback unsuccessful snapshots
>> >> >>> or continue them using create snapshot command.
>> >> >>>
>> >> >>> Option 2 - is to establish some heart beats between group client and
>> >> >>> image client. If group client fails to heart beat then image client
>> >> >>> unquiesces itself and continues normal operation.
>> >> >>>
>> >> >>> Option 3 - is to have a timeout for each image client. If group client
>> >> >>> fails to make a group snapshot within this timeout then we resume our
>> >> >>> normal operation informing group client of the fact.
>> >> >>>
>> >> >>> Which of these options do you prefer? Probably there are other options
>> >> >>> that I miss.
>> >> >>>
>> >> >>> Thanks,
>> >> >>> Victor.
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Jason
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Jason
>> >
>> >
>> >
>> > --
>> > Jason
>> >
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@xxxxxxxxxxxxxxx
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
