On Thu, Dec 03, 2015 at 09:11:34AM -0500, Poornima Gurusiddaiah wrote: > Hi, > > Brief Background: > ============ > For the below two features, we need ligfapi to take 2 other parameters from the applications for most number of fops. > http://review.gluster.org/#/c/12014/ > http://review.gluster.org/#/c/11980/ > > For leases to work as explained in the above doc, every file data read/write fop needs to be associated with a lease ID. This is specially required for Samba and NFS-Ganesha as they inturn serve other clients who request for leases. For Gluster to identify the end client (which is not Samba/NFS Ganesha) we need lease ID to be filled by Samba/NFS Ganesha. > > For mandatory locks feature to work as explained, every file data read/write fop needs to be associated with a lk_owner. In linux Kernel VFS takes care of filling the lk_ownere for the file system. In libgfapi case, the applications calling into libgfapi should be providing lk_owner with every fop. This is again required mainly for Samba and NFS Ganesha, as they serve multiple clients. > > Possible solutions: > ============= > 1. Modify all the required APIs to take 2 other parameter, lease ID and lk_owner. But that would mean backward compatibility issues and is a programming overhead for applications not interested in Leases and mandatory lock feature. > 2. Add an API called glfs_set_fop_attrs (lease ID, lk_owner) which works similar to glfs_set_uid(uid). The API sets a thread local storage (pthread_key) with the values provided, the further fops on that thread will pick the lease ID and lk_owner from the thread local storage (pthread_key). There are few minor details that needs to be worked out: > - In case of async API will end up using lease ID and lk_owner from wrong thread. > - unset lease ID and lk_owner after every fop to ensure there is no stale lease ID or lk_owner set? > - For fd based fops we can store the lease ID and lk_owner in glfd, so that the application writed need not set it for every fop. But for handle based fops lease ID and lk_owner needs to be set explicitly every-time. > > Solution 2 is more preferable except for that it adds overhead of calling another API, for the libgfapi users who intends to use these features. > A prototype of solution 2 can be found at http://review.gluster.org/#/c/12876/ > > Please let me know if you have any suggestions. We want to have some way to do compound operations in libgfapi. The basis for that would be re-usable for many other use-cases, including object orented APIs (PUT=CREATE+WRITE, GET=OPEN+READ, ..) and Kerberos where lk_owner can not be set in the RPC-header. A compound API in libgfapi would roughly look like a series of functions that build a structure. Once the structure has been built, it gets passed on to a call that actually starts the work. I am thinking of something like this (naming things without putting much thought in it): glfs_compound_call_t compound; glfs_compound_reply_t result; ret = glfs_compound_new (&compound); glfs_compound_fop (compound, OPEN, ...); glfs_compound_fop (compound, READ, ...); ret = glfs_compound_call (compound, &reply); The 'reply' would be a similar structure as the one built with glfs_compound_fop(), but with the replies of the FOPs. I think that is makes sense to create a SETLKOWNER and SETLEASEID (or something like that) 'pseudo FOP'. These pseudo FOPs would be able to carry some metadata that does not fit in the RPC-header. This makes it pretty simple to have one function setting the lk_owner or lease_id. Is there something that would not be solved by this? Now, I heard that there are some ideas for compound operations already. I have not seen any details about those yet, but am most interested! The above only shows the very minimal way that I think we can use immediately. But, it is possible to build much more complex compound operations, maybe someone would like if/else structures in there or other fancy statements. Please poke any developers that have ideas on compound operations ;-) Thanks, Niels
Attachment:
signature.asc
Description: PGP signature
_______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-devel
