On 9/21/21 3:24 AM, Jeff Layton wrote:
On Mon, 2021-09-20 at 22:32 +0800, Xiubo Li wrote:
On 9/18/21 1:19 AM, Jeff Layton wrote:
On Thu, 2021-09-16 at 18:02 +0800, Xiubo Li wrote:
For this I am a little curious why couldn't we cache truncate operations
when attr.ia_size >= isize ?
It seems to me that if attr.ia_size == i_size, then there is nothing to
change. We can just optimize that case away, assuming the client has
caps that ensure the size is correct.
From MDS side I didn't find any limitation why couldn't we optimize it.
Well...I think we do still need to change the mtime in that case. We
probably do still need to do a setattr or something that makes the MDS
set it to its current local time, even if the size doesn't change.
Since there hasn't any change for the file data, will change the 'mtime'
make sense here ? If so, then why in case the current client has the Fr
caps and sizes are the same it won't.
In this case I found in the MDS side, it will also update the 'ctime'
always even it will change nothing.
Based on the kclient code, for size changes after a write that extends a
file, it seems like Fw is sufficient to allow the client to buffer these
things.
Since the MDS will only allow us to increase the file size, just like
the mtime/ctime/atime, so even the write size has exceed current file
size, the Fw is sufficient.
Good.
For a truncate (aka setattr) operation, we apparently need Fx.
In case one client is truncating the file, if the new size is larger
than or equal to current size, this should be okay and will behave just
like normal write case above.
If the new size is smaller, the MDS will handle this in a different way.
When the MDS received the truncate request, it will first xlock the
filelock, which will switch the filelock state and in all these possible
interim or stable states, the Fw caps will be revoked from all the
clients, but the clients still could cache/buffer the file contents,
that means no client is allowed to change the size during the truncate
operation is on the way. After the truncate succeeds the MDS Locker will
issue_truncate() to all the clients and the clients will truncate the
caches/buffers, etc.
And also the truncate operations will always be queued sequentially.
It sort of makes sense, but the more I look over the existing code, the
less sure I am about how this is intended to work. I think before we
make any changes for fscrypt, we need to make sure we understand what's
there now.
So if my understanding is correct, the Fx is not a must for the truncate
operation.
Basically, when a truncate up or down comes in, we have to make a
determination of whether we can buffer that change, or whether we need
to do it synchronously.
The current kclient code bases that on whether it has Fx. I suspect
that ensures that no other client has Frw, which sounds like what we
probably want.
The Fx caps is only for the loner client and once the client has the Fx
caps it will always have the Fsrwcb at the same time. From the
mds/locker.cc I didn't find it'll allow that.
If current client has the Fx caps, so when there has another client is
trying to truncate the same file at the same time, the MDS will have to
revoke the Fx caps first and during which the buffered truncate will be
flushed and be finished first too. So when Fx caps is issued and new
size equals to the current size, why couldn't we buffer it ?
We need to prevent anyone from doing writes that might
extend the file at that time
Yeah, since the Fx is only for loner client, and all other clients will
have zero file caps. so it won't happen.
_and_ need to ensure that stat/statx for
the file size blocks until it's complete. ceph_getattr will issue a
GETATTR to the MDS if it doesn't have Fs in that case, so that should
be fine.
I assume that the MDS will never give out Fx caps to one client and Fs
to another.
Yeah, It won't happen.
What about Fw, though?
While for the Fw caps, it will allow it. If the filelock is in LOCK_MIX
state, the filelock maybe in this state if there at least one client
wants the Fw caps and some others want any of Fr and Fw caps.
Do Fx caps conflict with Frw the
same way as they do with Fs?
Yeah, it will be the same with Fs.
Thanks.
