RE: [ceph-users] Re: Unexpected behavior of directory mtime after being set explicitly

[Sandip Divekar <sandip.divekar@xxxxxxxxxxxxxxxxxx>]

Hi Chris /  Gregory,

 

Did you get a chance to investigate this issue ?  

 

Thanks and Regards

   Sandip Divekar

 

From: Sandip Divekar
Sent: Thursday, May 25, 2023 11:16 PM
To: Chris Palmer <chris.palmer@xxxxxxxxx>; ceph-users@xxxxxxx
Cc: dev@xxxxxxx; Gavin Lucas <gavin.lucas@xxxxxxxxxxxxxxxxxx>; Joseph Fernandes <joseph.fernandes@xxxxxxxxxxxxxxxxxx>; Simon Crosland <Simon.Crosland@xxxxxxxxxxxxxxxxxx>
Subject: RE: [ceph-users] Re: Unexpected behavior of directory mtime after being set explicitly

 

Hi Chris,

 

I think, you have missed one steps and that is to change mtime for directory explicitly. Please have a look at highlighted steps.

 

 

CEPHFS

===========================================================================

root@sds-ceph:/mnt/cephfs/volumes/_nogroup/test1/d5052b71-39ec-4d0a-9b0b-2091e1723538# mkdir dir1

root@sds-ceph:/mnt/cephfs/volumes/_nogroup/test1/d5052b71-39ec-4d0a-9b0b-2091e1723538# stat dir1

  File: dir1

  Size: 0               Blocks: 0          IO Block: 65536  directory

Device: 28h/40d Inode: 1099511714911  Links: 2

Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)

Access: 2023-05-24 11:09:25.260851345 +0530

Modify: 2023-05-24 11:09:25.260851345 +0530

Change: 2023-05-24 11:09:25.260851345 +0530

Birth: 2023-05-24 11:09:25.260851345 +0530

root@sds-ceph:/mnt/cephfs/volumes/_nogroup/test1/d5052b71-39ec-4d0a-9b0b-2091e1723538#  touch -m -d '26 Aug 1982 22:00' dir1

root@sds-ceph:/mnt/cephfs/volumes/_nogroup/test1/d5052b71-39ec-4d0a-9b0b-2091e1723538# stat dir1/

  File: dir1/

  Size: 0               Blocks: 0          IO Block: 65536  directory

Device: 28h/40d Inode: 1099511714911  Links: 2

Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)

Access: 2023-05-24 11:09:25.260851345 +0530

Modify: 1982-08-26 22:00:00.000000000 +0530

Change: 2023-05-24 11:10:04.881454967 +0530

Birth: 2023-05-24 11:09:25.260851345 +0530

root@sds-ceph:/mnt/cephfs/volumes/_nogroup/test1/d5052b71-39ec-4d0a-9b0b-2091e1723538# mkdir dir1/dir2

root@sds-ceph:/mnt/cephfs/volumes/_nogroup/test1/d5052b71-39ec-4d0a-9b0b-2091e1723538# stat dir1/

  File: dir1/

  Size: 1               Blocks: 0          IO Block: 65536  directory

Device: 28h/40d Inode: 1099511714911  Links: 3

Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)

Access: 2023-05-24 11:09:25.260851345 +0530

Modify: 1982-08-26 22:00:00.000000000 +0530

Change: 2023-05-24 11:10:19.141672220 +0530

Birth: 2023-05-24 11:09:25.260851345 +0530

root@sds-ceph:/mnt/cephfs/volumes/_nogroup/test1/d5052b71-39ec-4d0a-9b0b-2091e1723538#

 

 

Note : In a last step, it is expected that “Modify” time should change.

 

Thanks and Regards

   Sandip Divekar

 

 

 

From: Chris Palmer <chris.palmer@xxxxxxxxx>
Sent: Thursday, May 25, 2023 9:46 PM
To: Sandip Divekar <sandip.divekar@xxxxxxxxxxxxxxxxxx>; ceph-users@xxxxxxx
Cc: dev@xxxxxxx; Gavin Lucas <gavin.lucas@xxxxxxxxxxxxxxxxxx>; Joseph Fernandes <joseph.fernandes@xxxxxxxxxxxxxxxxxx>; Simon Crosland <Simon.Crosland@xxxxxxxxxxxxxxxxxx>
Subject: Re: [ceph-users] Re: Unexpected behavior of directory mtime after being set explicitly

 

***** EXTERNAL EMAIL *****

Hi Sandip

Ceph servers (debian11/ceph base with Proxmox installed on top - NOT the ceph that comes with Proxmox!):

ceph@pve1:~$ uname -a

Linux pve1 5.15.107-2-pve #1 SMP PVE 5.15.107-2 (2023-05-10T09:10Z) x86_64 GNU/Linux

ceph@pve1:~$ ceph version

ceph version 17.2.6 (d7ff0d10654d2280e08f1ab989c7cdf3064446a5) quincy (stable)

Fedora workstation. I waited until the minute had clicked over before doing each step:

[chris@rex mtime]$ uname -a

Linux rex.palmer 6.2.15-300.fc38.x86_64 #1 SMP PREEMPT_DYNAMIC Thu May 11 17:37:39 UTC 2023 x86_64 GNU/Linux

[chris@rex mtime]$ rpm -q ceph-common

ceph-common-17.2.6-2.fc38.x86_64

[chris@rex mtime]$ df .

Filesystem                                           1K-blocks       Used  Available Use% Mounted on

192.168.80.121,192.168.80.122,192.168.80.123:/data2 8589930496 4944801792 3645128704  58% /mnt/data2

[chris@rex mtime]$ mount|grep data2

systemd-1 on /mnt/data2 type autofs (rw,relatime,fd=61,pgrp=1,timeout=600,minproto=5,maxproto=5,direct,pipe_ino=22804)

192.168.80.121,192.168.80.122,192.168.80.123:/data2 on /mnt/data2 type ceph (rw,noatime,nodiratime,name=data2-rex,secret=<hidden>,fsid=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx,acl,_netdev,x-systemd.mount-timeout=30,x-systemd.automount,x-systemd.idle-timeout=600)

[chris@rex mtime]$ date; mkdir one; ls -ld one

Thu 25 May 16:57:28 BST 2023

drwxrwxr-x 2 chris groupname 0 May 25 16:57 one

[chris@rex mtime]$ date; touch one; ls -ld one

Thu 25 May 16:58:14 BST 2023

drwxrwxr-x 2 chris groupname 0 May 25 16:58 one

[chris@rex mtime]$ date; mkdir one/two; ls -ld one

Thu 25 May 16:59:26 BST 2023

drwxrwxr-x 3 chris groupname 1 May 25 16:59 one

I also repeated it with the test run on the ceph debian11 server, having mounted the cephfs filesystem on the ceph server - exactly the same result.

I then repeated it again on a pure debian11 ceph 17.2.6 cluster, using a debian11 client, and it also worked as expected.

All systems have latest patches applied.

Hope that helps
Chris

On 25/05/2023 15:57, Sandip Divekar wrote:

Hi Chris,

Kindly request you that follow steps given in previous mail and paste the output here.

The reason behind this request is that we have encountered an issue which is easily reproducible on

Latest version of both quincy and pacific, also we have thoroughly investigated the matter and we are certain that

No other factors are at play in this scenario.

Note :  We have used Debian 11 for testing.

sdsadmin@ceph-pacific-1:~$ uname -a

Linux ceph-pacific-1 5.10.0-10-amd64 #1 SMP Debian 5.10.84-1 (2021-12-08) x86_64 GNU/Linux

sdsadmin@ceph-pacific-1:~$ sudo ceph -v

ceph version 16.2.13 (5378749ba6be3a0868b51803968ee9cde4833a3e) pacific (stable)

Thanks for your prompt reply.

      Regards

   Sandip Divekar

-----Original Message-----

From: Chris Palmer <chris.palmer@xxxxxxxxx> 

Sent: Thursday, May 25, 2023 7:25 PM

To: ceph-users@xxxxxxx

Subject: [ceph-users] Re: Unexpected behavior of directory mtime after being set explicitly

***** EXTERNAL EMAIL *****

Hi Milind

I just tried this using the ceph kernel client and ceph-common 17.2.6 package in the latest Fedora kernel, against Ceph 17.2.6 and it worked perfectly...

There must be some other factor in play.

Chris

On 25/05/2023 13:04, Sandip Divekar wrote:

Hello Milind,

We are using Ceph Kernel Client.

But we found this same behavior while using Libcephfs library.

Should we treat this as a bug?  Or

Is there any existing bug for similar issue ?

Thanks and Regards,

   Sandip Divekar

From: Milind Changire <mchangir@xxxxxxxxxx>

Sent: Thursday, May 25, 2023 4:24 PM

To: Sandip Divekar <sandip.divekar@xxxxxxxxxxxxxxxxxx>

Cc: ceph-users@xxxxxxx; dev@xxxxxxx

Subject: Re: [ceph-users] Unexpected behavior of directory mtime after 

being set explicitly

***** EXTERNAL EMAIL *****

Sandip,

What type of client are you using ?

kernel client or fuse client ?

If it's the kernel client, then it's a bug.

FYI - Pacific and Quincy fuse clients do the right thing

On Wed, May 24, 2023 at 9:24 PM Sandip Divekar <sandip.divekar@xxxxxxxxxxxxxxxxxx<mailto:sandip.divekar@xxxxxxxxxxxxxxxxxx>> wrote:

Hi Team,

I'm writing to bring to your attention an issue we have encountered with the "mtime" (modification time) behavior for directories in the Ceph filesystem.

Upon observation, we have noticed that when the mtime of a directory 

(let's say: dir1) is explicitly changed in CephFS, subsequent additions of files or directories within 'dir1' fail to update the directory's mtime as expected.

This behavior appears to be specific to CephFS - we have reproduced this issue on both Quincy and Pacific.  Similar steps work as expected in the ext4 filesystem amongst others.

Reproduction steps:

1. Create a directory - mkdir dir1

2. Modify mtime using the touch command - touch dir1 3. Create a file 

or directory inside of 'dir1' - mkdir dir1/dir2 Expected result:

mtime for dir1 should change to the time the file or directory was 

created in step 3 Actual result:

there was no change to the mtime for 'dir1'

Note : For more detail, kindly find the attached logs.

Our queries are :

1. Is this expected behavior for CephFS?

2. If so, can you explain why the directory behavior is inconsistent depending on whether the mtime for the directory has previously been manually updated.

Best Regards,

   Sandip Divekar

Component QA Lead SDET.

_______________________________________________

ceph-users mailing list -- 

ceph-users@xxxxxxx<mailto:ceph-users@xxxxxxx>

To unsubscribe send an email to 

ceph-users-leave@xxxxxxx<mailto:ceph-users-leave@xxxxxxx>

--

Milind

_______________________________________________

ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an 

email to ceph-users-leave@xxxxxxx

_______________________________________________

ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx

_______________________________________________

ceph-users mailing list -- ceph-users@xxxxxxx

To unsubscribe send an email to ceph-users-leave@xxxxxxx

 

_______________________________________________
Dev mailing list -- dev@xxxxxxx
To unsubscribe send an email to dev-leave@xxxxxxx