Hi Greg,
I don't read server code. But from my test: I use ceph-fuse to mount
/mnt/test1 and /mnt/test2. If I open a file with Fr and don't release
the cap in /mnt/test1 . I can write the same file successfully in
/mnt/test2. So it seems client1 hold Fs and client2 can still require
Fw. Is it right?
Marvin
On Wed, Nov 7, 2018 at 3:14 AM Gregory Farnum <gfarnum@xxxxxxxxxx> wrote:
>
> On Tue, Nov 6, 2018 at 4:16 AM Jeff Layton <jlayton@xxxxxxxxxx> wrote:
> >
> > On Tue, 2018-11-06 at 15:11 +0800, Marvin Zhang wrote:
> > > Thanks for your explanation.
> > > Here is the reason why I want to clearfy the difference between Fs and Fr
> > > 1. When I open and read a file, Client::_open() will send a
> > > MetaRequest(CEPH_MDS_OP_OPEN) message to Server.
> >
> > I'm assuming you close the file after the read here?
> >
> > > 2. If I try to open this file during 5s, Client::_open() will not send
> > > MetaRequest to MDS.
> > > 3. If I try to open this file after 5s, Client::_open() will send
> > > MetaRequest to MDS, even though the file inode and dentry informations
> > > exist in client cache.
> > >
> > > And here is my question:
> > > 1. client_caps_release_delay is 5s by default. If I increase this
> > > number, client will not send MetaRequest(CEPH_MDS_OP_OPEN) too many
> > > times. Is it OK that I set this value very big, e.g 1h? Is there any
> > > bad influence?
> > > 2. in Client::check_caps()
> > > {
> > > wanted = in->caps_wanted(); //it will be 0 after release file
> > > if (wanted)
> > > retain |= CEPH_CAP_ANY;
> > > else
> > > retain |= CEPH_CAP_ANY_SHARED; //why exclude FILE_RD??
> > > //this will cause drop FILE_RD in below code, so in next
> > > Client::_open() see the cap don't have Fr, it will send a
> > > MetaRequest(CEPH_MDS_OP_OPEN) to MSD
> > > }
> > > My question is : Is is OK to set retain |= (CEPH_CAP_ANY_SHARED |
> > > CEPH_CAP_FILE_RD) ?
> > >
> >
> > That sounds reasonable at first glance. There are no real hard and fast
> > rules in the cap handling code. It really comes down to heuristics and
> > trying to minimize later cap revokes.
> >
> > Doing what you suggest shouldn't break anything, as far as I can tell.
> > It's just that in some (possibly common) situations, your MDS might end
> > up having to send more cap revokes when there is competing access. Maybe
> > it has to do that already for Fs though?
>
> Yeah, I'd be concerned less about it breaking things (though it's
> possible for the Client and MDS to get in a capabilities fight, I
> don't think this would cause it to do so) and more about what happens
> when conflicting activity is trying to occur. If a writer comes in and
> is doing active IO but this client is trying to hold Fr caps, it will
> prevent the writer from holding the "Fb" cap that lets it buffer up
> writes before dispatching them to the OSD. IIRC, holding on to the Fr
> cap will also make the MDS more likely to let the client keep the Fc
> (cache) capability, which means if a writer comes along it needs to do
> another cap revoke. Etc etc.
> -Greg
