On 07/21/2015 02:49 PM, Poornima Gurusiddaiah wrote:
Hi Shyam,
Please find my reply inline.
Rgards,
Poornima
----- Original Message -----
From: "Ira Cooper" <icooper@xxxxxxxxxx>
To: "Shyam" <srangana@xxxxxxxxxx>
Cc: "Gluster Devel" <gluster-devel@xxxxxxxxxxx>
Sent: Saturday, July 18, 2015 4:09:30 AM
Subject: Re: Lease-lock Design notes
1. Yes, it is intentional. The internals of gluster should be able to
lease-lks. We discussed using them in the read ahead translator and the
write behind translator.
2. This has been discussed, and proposed, but there is actually a need for a
lease fop also, because clients can request the "renewal" or "reinstatement"
of a lease. (Actually for Samba having it all be one call and atomic is
very interesting.)
3. This I can't answer... I haven't been in the most recent discussions. But
the intent of this work, when I started, was to be useful to the whole of
Gluster, not just Samba or Ganesha.
Thanks,
-Ira
----- Original Message -----
Hi,
I have some questions or gaps in understanding the current lease design,
request some clarification on the same.
1) Is there a notion of a *gluster* client holding a lease on a file/dir?
Yes, a lease is granted to a gluster client on a file, dir is not yet implemented, but is on cards.
The lease can be requestd from any xlator, and is granted for the whole client.
- As per your Barcelona gluster summit presentation the client side
caching needs this notion
- DHT/EC/AFR can leverage such a lease to prevent eager or any form of
locking when attempting to hold consistency of data being operated on
- Please note that this requires not each xlator requesting and
holding the lease, but a *gluster* client holding the lease, assuming
that one can do local in memory locking to prevent different
fd/connection/applications performing operations on the same file
against the *same* gluster client and not have to coordinate this with
the brick
Apart from the in memory locking (for different fds) which you have
mentioned, the other complexity involved here (which I can think of atm)
is that unlike existing inodelk/entry_lk locks, leases can be recalled
and revoked by server. We need to consider the amount of time needed by
each of these xlators to finish their tasks(may include re-acquiring
locks) before they send recall_lease event to their parent xlators. Or
(as already mentioned in the NFS protocol), we need a way to let server
increase the recall_lease timeout dynamically if the client is
diligently flushing the data which I think is doable. But the switch
between leases<->locks sounds racy :)
That said as Poornima has mentioned below, currently we have started
with requesting & handling leases at application layer. Later we shall
explore more on having a client-side xlator to handle leases and cache
data.
Thanks,
Soumya
I see some references to this in the design but wanted to understand if
the thought is there.
The initial thought of having leases in Gluster was to support Multiprotocol access,
aprt from this another use case we saw was, having a Gluster client cache xlator
which caches taking leases. Yes, DHT/EC/AFR also could leverage leases, but i m not
sure if leases can replace eager locks.
IOW, if an NFS client requests a lease, Ganesha requests the lease from
Gluster, in which case the gfapi client that requested the lease first
gets the lease, and then re-leases it to Ganesha, now Ganesha is free to
lease it to any client on its behalf and recall leases etc. as the case
may be and the gluster client does not care. When another gluster client
(due to Ganesha or otherwise (say self heal or rebalance)) attempts to
get a lease, that is when the lease is broken all across.
Is my understanding right, is the design along these lines?
Yes, that is right, any conflicts between its(ganesha/Samba) clients should be
resolved by the Ganesha/Samba server. Gluster server will handle the conflicts
across gluster clients.
2) Why is the lease not piggy backed with the open? Why 2 network FOPs
instead of 1 in this case? How can we compound this lease with other
requests? Or do you have thoughts around the same?
Isn't NFS and SMB requesting leases with its open calls
Our initial thought was to overload lk call to request for lease, and also
support open+lease. But the problems with fd based leases were:
- Doesn't go well with the NFS world, where handles are created and
delegations are granted followed by multiple open/read/write/close
on that fd. Hence lease is more conveniently associated with handles
than fds, in NFS.
- Anonymous fds are used by NFSV3 and pnfs to maintain statelessness,
aonymous fds means there is no open fd on the file, the backend opens
read/writes to the file and closes. These anonfds makes it harder to get
the lease conflict check right, and we may break leases when it is not
necessary.
- The lease might have to live longer than fd (handle based leases, i.e.
when persistent/durable handles will be used instead of fd).
That said, we could even now overload open call to request for lease,
but it will be associated with the inode and the client and not the fd,
for the above said reasons.
3) If there were discussions around some designs that were rejected,
could you also update the document with the same, so the one can
understand the motivation behind the current manner of implementing leases.
Yes sure, we are updating the leases.md, will send it at the earliest.
Thanks,
Shyam
On 04/16/2015 07:37 AM, Soumya Koduri wrote:
Hi,
Below link contains the lease-lock design notes (as per the latest
discussion we had). Thanks to everyone involved (CC'ed).
http://www.gluster.org/community/documentation/index.php/Features/Upcall-infrastructure#delegations.2Flease-locks
Kindly go through the same and provide us your inputs.
Thanks,
Soumya
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-devel
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-devel
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-devel
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-devel
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-devel
