On 18/11/2022 02:02, Luís Henriques wrote:
Xiubo Li <xiubli@xxxxxxxxxx> writes:
On 17/11/2022 16:03, Xiubo Li wrote:
On 16/11/2022 23:37, Luís Henriques wrote:
When setting a directory's crypt context, __ceph_dir_clear_complete() needs
to be used otherwise, if it was complete before, any old dentry that's still
around will be valid.
Signed-off-by: Luís Henriques <lhenriques@xxxxxxx>
---
Hi!
Here's a simple way to trigger the bug this patch is fixing:
# cd /cephfs
# ls mydir
nKRhofOAVNsAwVLvDw7a0c9ypsjbZfK3n0Npnmni6j0
# ls mydir/nKRhofOAVNsAwVLvDw7a0c9ypsjbZfK3n0Npnmni6j0/
Cyuer5xT+kBlEPgtwAqSj0WK2taEljP5vHZ,D8VXCJ8
u+46b2XVCt7Obpz0gznZyNLRj79Q2l4KmkwbKOzdQKw
# fscrypt unlock mydir
# touch /mnt/test/mydir/mysubdir/file
touch: cannot touch '/mnt/test/mydir/mysubdir/file': No such file or
directory
fs/ceph/crypto.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/fs/ceph/crypto.c b/fs/ceph/crypto.c
index 35a2ccfe6899..dc1557967032 100644
--- a/fs/ceph/crypto.c
+++ b/fs/ceph/crypto.c
@@ -87,6 +87,10 @@ static int ceph_crypt_get_context(struct inode *inode,
void *ctx, size_t len)
return -ERANGE;
memcpy(ctx, cfa->cfa_blob, ctxlen);
+
+ /* Directory isn't complete anymore */
+ if (S_ISDIR(inode->i_mode) && __ceph_dir_is_complete(ci))
+ __ceph_dir_clear_complete(ci);
Hi Luis,
Good catch!
BTW, why do this in the ceph_crypt_get_context() ? As my understanding is that
we should mark 'mydir' as incomplete when unlocking it. While as I remembered
the unlock operation will do:
Step1: get_encpolicy via 'mydir' as ctx
Step2: rm_enckey of ctx from the superblock
Sorry, it should be add_enckey.
Since I am still running the test cases for the file lock patches, so I didn't
catch logs to confirm the above steps yet.
If I am right IMO then we should mark the dir as incomplete in the Step2
instead, because for non-unlock operations they may also do the Step1.
Your patch will work. But probably we should do this just around
__fscrypt_prepare_readdir() or fscrypt_prepare_readdir() instead ? We need to
detect that once the 'inode->i_crypt_info' changed then mark the dir as
incomplete.
For now for the lock operation it will evict the inode, which will help do this
for us already. But for unlock case, we need to handle it by ourself.
OK, that makes sense and to be honest I thought that there should be
another place for doing this. Unfortunately, I didn't found it: in the
test case I have the fscrypt_prepare_readdir() isn't called:
# cd /cephfs
# ls mydir
nKRhofOAVNsAwVLvDw7a0c9ypsjbZfK3n0Npnmni6j0
# ls mydir/nKRhofOAVNsAwVLvDw7a0c9ypsjbZfK3n0Npnmni6j0/
Cyuer5xT+kBlEPgtwAqSj0WK2taEljP5vHZ,D8VXCJ8 u+46b2XVCt7Obpz0gznZyNLRj79Q2l4KmkwbKOzdQKw
At this point readdir was executed, of course. And
__ceph_dir_set_complete() is also used to indicate that we have the full
contents. However, executing the following commands won't result in any
new readdir():
# fscrypt unlock mydir
# touch /mnt/test/mydir/mysubdir/file
and since the encryption key is set at the sb level, I couldn't find a way
to detect changes in inode->i_crypt_info. ceph_d_revalidate() is invoked
but at that point I don't thing we have a way to know what is changing.
Any ideas?
# ls mydir/
zy94Zt01M90xwYq+nxJsEvea+HYq49mqVgrUBkYrJAU
# ls mydir/zy94Zt01M90xwYq+nxJsEvea+HYq49mqVgrUBkYrJAU/
24iNa8ICYc6nZZIkL,4n1sOtp9KiaTGtpq8PCZDQ6LU
ivG4rQYzg6YENagbTZaV2kh2sVouEdlkmBZPfgVDl48
I have added the debug logs in all the places just like:
@@ -784,7 +786,9 @@ static struct dentry *ceph_lookup(struct inode *dir,
struct dentry *dentry,
return ERR_PTR(-ENAMETOOLONG);
if (IS_ENCRYPTED(dir)) {
+ printk("%s fscrypt_has_encryption_key(dir): %d ======\n",
__func__, fscrypt_has_encryption_key(dir));
err = __fscrypt_prepare_readdir(dir);
+ printk("%s fscrypt_has_encryption_key(dir): %d ======\n",
__func__, fscrypt_has_encryption_key(dir));
if (err)
return ERR_PTR(err);
if (!fscrypt_has_encryption_key(dir)) {
# fscrypt unlock mydir
Enter custom passphrase for protector "l":
"mydir" is now unlocked and ready for use.
And after "mydir/" was unlocked and then when doing:
# touch mydir/dir/file2
And when lookup the "dir/" dentry I can see the "mydir/" encryption key
changed:
709 <7>[79125.023676] ceph: __ceph_caps_issued_mask ino 0x10000000004
cap 000000004dc11892 issued pAsLsXsFs (mask As)
710 <7>[79125.023687] ceph: __touch_cap 000000007071b095 cap
000000004dc11892 mds0
711 <7>[79125.023823] ceph: lookup 000000007071b095 dentry
00000000f97501b2 'dir'
712 <4>[79125.023838] ceph_lookup fscrypt_has_encryption_key(dir): 0 ======
713 <4>[79125.024186] ceph_lookup fscrypt_has_encryption_key(dir): 1 ======
714 <7>[79125.024194] ceph: dir 000000007071b095 flags are 0x0
715 <7>[79125.024269] ceph: do_request on 00000000a93fafef
I am thinking could we just make __fscrypt_prepare_readdir(), which will
return 0 when the key is already set or successfully set, to return 1
instead of 0 to mark that the key changed ?
Thanks!
- Xiubo
Cheers,
