Hi Xiubo, I will update the case. I'm afraid this will have to wait a little bit though. I'm too occupied for a while and also don't have a test cluster that would help speed things up. I will update you, please keep the tracker open. Best regards, ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14 ________________________________________ From: Xiubo Li <xiubli@xxxxxxxxxx> Sent: Tuesday, December 5, 2023 1:58 AM To: Frank Schilder; Gregory Farnum Cc: ceph-users@xxxxxxx Subject: Re: Re: ceph fs (meta) data inconsistent Frank, By using your script I still couldn't reproduce it. Locally my python version is 3.9.16, and I didn't have other VMs to test python other versions. Could you check the tracker to provide the debug logs ? Thanks - Xiubo On 12/1/23 21:08, Frank Schilder wrote: > Hi Xiubo, > > I uploaded a test script with session output showing the issue. When I look at your scripts, I can't see the stat-check on the second host anywhere. Hence, I don't really know what you are trying to compare. > > If you want me to run your test scripts on our system for comparison, please include the part executed on the second host explicitly in an ssh-command. Running your scripts alone in their current form will not reproduce the issue. > > Best regards, > ================= > Frank Schilder > AIT Risø Campus > Bygning 109, rum S14 > > ________________________________________ > From: Xiubo Li <xiubli@xxxxxxxxxx> > Sent: Monday, November 27, 2023 3:59 AM > To: Frank Schilder; Gregory Farnum > Cc: ceph-users@xxxxxxx > Subject: Re: Re: ceph fs (meta) data inconsistent > > > On 11/24/23 21:37, Frank Schilder wrote: >> Hi Xiubo, >> >> thanks for the update. I will test your scripts in our system next week. Something important: running both scripts on a single client will not produce a difference. You need 2 clients. The inconsistency is between clients, not on the same client. For example: > Frank, > > Yeah, I did this with 2 different kclients. > > Thanks > >> Setup: host1 and host2 with a kclient mount to a cephfs under /mnt/kcephfs >> >> Test 1 >> - on host1: execute shutil.copy2 >> - execute ls -l /mnt/kcephfs/ on host1 and host2: same result >> >> Test 2 >> - on host1: shutil.copy >> - execute ls -l /mnt/kcephfs/ on host1 and host2: file size=0 on host 2 while correct on host 1 >> >> Your scripts only show output of one host, but the inconsistency requires two hosts for observation. The stat information is updated on host1, but not synchronized to host2 in the second test. In case you can't reproduce that, I will append results from our system to the case. >> >> Also it would be important to know the python and libc versions. We observe this only for newer versions of both. >> >> Best regards, >> ================= >> Frank Schilder >> AIT Risø Campus >> Bygning 109, rum S14 >> >> ________________________________________ >> From: Xiubo Li <xiubli@xxxxxxxxxx> >> Sent: Thursday, November 23, 2023 3:47 AM >> To: Frank Schilder; Gregory Farnum >> Cc: ceph-users@xxxxxxx >> Subject: Re: Re: ceph fs (meta) data inconsistent >> >> I just raised one tracker to follow this: >> https://tracker.ceph.com/issues/63510 >> >> Thanks >> >> - Xiubo >> >> > > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |