I get really strange timings depending on kernel version; see below. Did the patch of the kernel client get lost? The only difference between gnosis and smb01 is that gnosis is physical and smb01 is a KVM. Both have direct access to the client network and use the respective kernel clients. Timings of moving a 3.2G file around, crossing quota realms: [root@gnosis ~]# uname -r 5.9.9-1.el7.elrepo.x86_64 [root@gnosis ~]# time mv /mnt/cephfs/A/testfile.bin /mnt/cephfs/B/ real 0m0.013s user 0m0.000s sys 0m0.001s [root@gnosis ~]# time mv /mnt/cephfs/B/testfile.bin /mnt/cephfs/A/ real 0m0.008s user 0m0.000s sys 0m0.001s [root@gnosis ~]# time cp /mnt/cephfs/A/testfile.bin /mnt/cephfs/B/ real 0m16.807s user 0m0.000s sys 0m2.150s [root@gnosis ~]# time \rm /mnt/cephfs/B/testfile.bin real 0m0.004s user 0m0.000s sys 0m0.001s [root@smb01 ~]# uname -r 5.12.12-1.el7.elrepo.x86_64 [root@smb01 ~]# time mv /mnt/cephfs/A/testfile.bin /mnt/cephfs/B/ real 0m8.545s user 0m0.004s sys 0m1.999s [root@smb01 ~]# time mv /mnt/cephfs/B/testfile.bin /mnt/cephfs/A/ real 0m2.986s user 0m0.003s sys 0m1.644s [root@smb01 ~]# time cp /mnt/cephfs/A/testfile.bin /mnt/cephfs/B/ real 0m3.564s user 0m0.003s sys 0m1.671s [root@smb01 ~]# time \rm /mnt/cephfs/B/testfile.bin real 0m0.022s user 0m0.001s sys 0m0.000s [root@smb01 ~]# uname -r 5.4.127-1.el7.elrepo.x86_64 [root@smb01 ~]# time mv /mnt/cephfs/A/testfile.bin /mnt/cephfs/B/ real 0m17.268s user 0m0.000s sys 0m2.010s [root@smb01 ~]# time mv /mnt/cephfs/B/testfile.bin /mnt/cephfs/A/ real 0m3.404s user 0m0.002s sys 0m1.635s [root@smb01 ~]# time cp /mnt/cephfs/A/testfile.bin /mnt/cephfs/B/ real 0m3.569s user 0m0.002s sys 0m1.825s [root@smb01 ~]# time \rm /mnt/cephfs/B/testfile.bin real 0m0.031s user 0m0.001s sys 0m0.001s [root@smb01 ~]# uname -r 3.10.0-1160.31.1.el7.x86_64 [root@smb01 ~]# time mv /mnt/cephfs/A/testfile.bin /mnt/cephfs/B/ real 0m25.695s user 0m0.003s sys 0m1.936s [root@smb01 ~]# time mv /mnt/cephfs/B/testfile.bin /mnt/cephfs/A/ real 0m3.173s user 0m0.001s sys 0m2.302s [root@smb01 ~]# time cp /mnt/cephfs/A/testfile.bin /mnt/cephfs/B/ real 0m2.674s user 0m0.004s sys 0m2.057s [root@smb01 ~]# time \rm /mnt/cephfs/B/testfile.bin real 0m0.016s user 0m0.000s sys 0m0.001s ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14 ________________________________________ From: Frank Schilder <frans@xxxxxx> Sent: 22 June 2021 11:15:06 To: ceph-users@xxxxxxx Subject: ceph fs mv does copy, not move Dear all, some time ago I reported that the kernel client resorts to a copy instead of move when moving a file across quota domains. I was told that the fuse client does not have this problem. If enough space is available, a move should be a move, not a copy. Today, I tried to move a large file across quota domains testing botn, the kernel- and the fuse client. Both still resort to a copy even though this issue was addressed quite a while ago (https://lists.ceph.io/hyperkitty/list/ceph-users@xxxxxxx/thread/44AEIHNEGKV4VGCARRTARGFZ264CR4T7/#XY7ZCE3KWHI4QSUNZHDWL3QZQFOHXRQW). The versions I'm using are (CentOS 7) # yum list installed | grep ceph-fuse ceph-fuse.x86_64 2:13.2.10-0.el7 @ceph # uname -r 3.10.0-1160.31.1.el7.x86_64 Any suggestions how to get this to work? I have to move directories containing 100+ TB. Many thanks, ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14 _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |