> -----Original Message----- > From: Trond Myklebust [mailto:Trond.Myklebust@xxxxxxxxxx] > Sent: mardi 1 mars 2011 13:16 > To: Gardere, Daniel > Cc: bfields@xxxxxxxxxxxx; rees@xxxxxxxxx; bhalevy@xxxxxxxxxxx; linux- > nfs@xxxxxxxxxxxxxxx; honey@xxxxxxxxxxxxxx > Subject: RE: [PATCH] zero out delegation in the inode after it has been > returned > > On Tue, 2011-03-01 at 04:52 -0500, daniel.gardere@xxxxxxx wrote: > > It is EMC server. > > > > The server returns the delegation to the OPEN CLAIM_DELEGATE_CUR > operation. At this point the delegation is still valid. This is far > before the DELEGRETURN; so this doesn't explain why > > the delegation is used in SETATTR (frame 5596) after the DELEGRETURN > (frame 5589, reply in 5592). > > It explains _exactly_ what is happening... > > The server hands out a delegation in step 4. of the recovery process, > which causes the client to attach a new delegation to the > nfsi->delegation. You can argue that the client shouldn't be doing that > while it is in the middle of a recovery process, but as far as our > client is concerned, it doesn't expect the server to violate the > protocol in this manner. > OK I understand the situation. > > Returning a delegation to OPEN CLAIM_DELEGATE_CUR is probably > useless, as the client already has this delegation. But, I don't think > that it is illegal. > > If this creates the confusion, we can avoid returning the delegation > in this case. > > See section 8.1.8 (as I noted in my followup email). The server is not > supposed to send a delegation as part of a CLAIM_DELEGATE_CUR request. > You are violating the spec. > Agreed. I missed this point in the RFC. > Sending a delegation more than once is neither necessary, nor is it > really a good idea even if the client sends multiple CLAIM_NULL calls. > In fact it is a recipe for creating races... > The reasons are: > > * Unlike layouts, there is no 'forgetful' model for delegations. > Once the client receives a delegation, it is expected to keep > track of that delegation and return it when it no longer needs > it. > * In NFSv4.0, there is no good way of dealing with races between > OPEN and CB_RECALL. If the server is handing out a delegation, > but then needs to recall it for some reason, then it has no way > of ensuring that the client processes the CB_RECALL _after_ the > reply to the OPEN in which the delegation was handed out for > the > second time. > > IOW: there is a danger of something like the following race occurring: > > Client Server > > Send OPEN(CLAIM_NULL) > hands out delegation > process open result > client now thinks it > holds a delegation > > Server now thinks client > holds a delegation > > Send OPEN(CLAIM_NULL) > hands out delegation again > Send CB_RECALL > process CB_RECALL > Send DELEGRETURN > > process open result > client now thinks it > holds a delegation > > Server thinks client doesn't hold > a delegation. > > This case is a little bit different. I am going to make sure that delegations are returned only once. > > > -----Original Message----- > > From: linux-nfs-owner@xxxxxxxxxxxxxxx [mailto:linux-nfs- > owner@xxxxxxxxxxxxxxx] On Behalf Of J. Bruce Fields > > Sent: mardi 1 mars 2011 05:10 > > To: Trond Myklebust > > Cc: Jim Rees; Benny Halevy; linux-nfs@xxxxxxxxxxxxxxx; peter honeyman > > Subject: Re: [PATCH] zero out delegation in the inode after it has > been returned > > > > On Mon, Feb 28, 2011 at 07:00:33PM -0500, Trond Myklebust wrote: > > > On Mon, 2011-02-28 at 18:22 -0500, Jim Rees wrote: > > > > Trond Myklebust wrote: > > > > > > > > The procedure for returning delegations is supposed to work as > follows: > > > > > > > > 1. Remove the nfsi->delegation so that it is no longer > visible to > > > > new open() requests > > > > 2. write back any dirty data to the server > > > > 3. Reclaim any locks > > > > 4. Reclaim any open stateids (using CLAIM_DELEGATE_CUR) > > > > 5. delegreturn > > > > > > > > While there may indeed be the odd READ or WRITE that races > between 4. > > > > and 5., so that the server receives the delegation stateid > after the > > > > delegreturn, that shouldn't matter: the server returns an > error, and the > > > > client should retry using the new open stateid. > > > > > > > > What is failing to work correctly here? > > > > > > > > That helps, thanks. I'll see if I can figure out what's going > wrong. > > > > > > > > At the server, I see a delegreturn followed by a setattr using > the returned > > > > stateid. The server returns BAD_STATEID. I'll look to see if > the client > > > > then retries. > > > > > > > > At the client, I see a non-null nfsi->delegation after the > delgreturn, and > > > > the application gets a EIO. > > > > > > That's because your server is confusing the hell out of us all by > giving > > > out a delegation as part of the reply (in frame 5328) to the > > > OPEN(CLAIM_DELEGATE_CUR) in frame 5325. > > > > > > IOW: the client is in the process of returning the delegation, and > asks > > > to trade in the delegation stateid into an open stateid, then the > server > > > replies with an open stateid, and the delegation stateid... > > > > Which server is this, Jim? > > > > (I checked the Linux server quickly and it doesn't look like it > should > > do this....) > > > > --b. > > -- > > To unsubscribe from this list: send the line "unsubscribe linux-nfs" > in > > the body of a message to majordomo@xxxxxxxxxxxxxxx > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > > > -- > Trond Myklebust > Linux NFS client maintainer > > NetApp > Trond.Myklebust@xxxxxxxxxx > www.netapp.com > ÿô.nlj·Ÿ®‰†+%ŠË±é¥Šwÿº{.nlj·¥Š{±þwìíè jg¬±¨ ¶‰šŽŠÝjÿ¾«þG«é¸ ¢·¦j:+v‰¨Šwèm¶Ÿÿþø®w¥þŠà£¢·hšâÿ†Ù
