I've been trying to understand how LAYOUTRETURN is used in pNFS, primarily because our SLE12-SP1 kernel (based on 3.12) appears to have a very different opinion than some Netapp filers. My reading of RFC-5661 suggests that the client needs to call LAYOUTRETURN for every layout that it received from the server. A single LAYOUTRETURN can cover a whole file or a whole filesystem, so it doesn't need to be 1-for-1, but there is no implicit return. However RFC-5663 contains the text A LAYOUTRETURN operation represents an explicit release of resources by the client, usually done for the purpose of avoiding unnecessary CB_LAYOUTRECALL operations in the future. This seems to imply that LAYOUTRETURN is only an optimisation. If you don't want to avoid CB_LAYOUTRECALL, there is not much call for LAYOUTRETURN. It seems to suggest (without explicitly saying) that the CB_LAYOUTRECALL will effect the return of a layout without the client explicitly sending LAYOUTRETURN in response. RFC-5661 says LAYOUTRETURN does need to be sent in response. The code in 3.12 doesn't send LAYOUTRETURN in response to CB_LAYOUTRECALL, nor does it send LAYOUTRETURN when it closes a file marked as "return layouts on close". The one place I have seen evidence of it returning layouts is when a file is unlinked, though I think there are others (chmod, IO error). The current upstream code seems to call LAYOUTRETURN more correctly, but it is hard to be sure because I couldn't find a commit which acknowledged the specific problem and corrected it - just commits that claim to be making improvements and avoiding races and things like that. Questions: - Am I correct that all layouts need to be explicitly returned by the client, and so the text from RFC-5663 is misleading? - If so, what is the earliest kernel that is believed to correctly return layouts in response to CB_LAYOUTRECALL, or a 'roc' file being closed? I was advised that Netapp are considering a change (netapp issue 955835): An enhancement will be added in future versions of Ontap to clear out the corresponding layout states after a file has been closed in the event the client does not return them. This sounds like a mistake, unless "clear out" means "send CB_LAYOUTRECALL for". Should we advice Netapp against this? Thanks, NeilBrown
Attachment:
signature.asc
Description: PGP signature
