On Tue, Nov 6, 2018 at 4:16 AM Jeff Layton <jlayton@xxxxxxxxxx> wrote:
>
> On Tue, 2018-11-06 at 15:11 +0800, Marvin Zhang wrote:
> > Thanks for your explanation.
> > Here is the reason why I want to clearfy the difference between Fs and Fr
> > 1. When I open and read a file, Client::_open() will send a
> > MetaRequest(CEPH_MDS_OP_OPEN) message to Server.
>
> I'm assuming you close the file after the read here?
>
> > 2. If I try to open this file during 5s, Client::_open() will not send
> > MetaRequest to MDS.
> > 3. If I try to open this file after 5s, Client::_open() will send
> > MetaRequest to MDS, even though the file inode and dentry informations
> > exist in client cache.
> >
> > And here is my question:
> > 1. client_caps_release_delay is 5s by default. If I increase this
> > number, client will not send MetaRequest(CEPH_MDS_OP_OPEN) too many
> > times. Is it OK that I set this value very big, e.g 1h? Is there any
> > bad influence?
> > 2. in Client::check_caps()
> > {
> > wanted = in->caps_wanted(); //it will be 0 after release file
> > if (wanted)
> > retain |= CEPH_CAP_ANY;
> > else
> > retain |= CEPH_CAP_ANY_SHARED; //why exclude FILE_RD??
> > //this will cause drop FILE_RD in below code, so in next
> > Client::_open() see the cap don't have Fr, it will send a
> > MetaRequest(CEPH_MDS_OP_OPEN) to MSD
> > }
> > My question is : Is is OK to set retain |= (CEPH_CAP_ANY_SHARED |
> > CEPH_CAP_FILE_RD) ?
> >
>
> That sounds reasonable at first glance. There are no real hard and fast
> rules in the cap handling code. It really comes down to heuristics and
> trying to minimize later cap revokes.
>
> Doing what you suggest shouldn't break anything, as far as I can tell.
> It's just that in some (possibly common) situations, your MDS might end
> up having to send more cap revokes when there is competing access. Maybe
> it has to do that already for Fs though?
Yeah, I'd be concerned less about it breaking things (though it's
possible for the Client and MDS to get in a capabilities fight, I
don't think this would cause it to do so) and more about what happens
when conflicting activity is trying to occur. If a writer comes in and
is doing active IO but this client is trying to hold Fr caps, it will
prevent the writer from holding the "Fb" cap that lets it buffer up
writes before dispatching them to the OSD. IIRC, holding on to the Fr
cap will also make the MDS more likely to let the client keep the Fc
(cache) capability, which means if a writer comes along it needs to do
another cap revoke. Etc etc.
-Greg
