On Thu, Mar 26, 2020 at 9:13 AM Frank Schilder <frans@xxxxxx> wrote: > > Dear all, > > yes, this is it, quotas. In the structure A/B/ there was a quota set on A. Hence, B was moved out of this zone and this does indeed change mv to be a cp+rm. > > The obvious follow-up. What is the procedure for properly moving data as an administrator? Do I really need to unset quotas, do the move and set quotas back again? Ah-ha, this is a difference between the userspace and kernel clients. :( The kernel client always returns EXDEV if crossing "quota realms" (different quota roots). I'm not sure why it behaves that way as userspace is different: * If fhere is a quota in the target directory, and * If the target directory's existing data, plus the file(s) being moved, exceed the target directory's quota, then userspace returns EXDEV/EDQUOT. This seems like the right kernel behavior as well... Jeff, is that a known issue for some reason? Should we make a new bug? :) -Greg > > Best regards, > > ================= > Frank Schilder > AIT Risø Campus > Bygning 109, rum S14 > > ________________________________________ > From: Frank Schilder <frans@xxxxxx> > Sent: 26 March 2020 16:35:07 > To: Gregory Farnum; Beocat KSU > Cc: ceph-users > Subject: Re: Move on cephfs not O(1)? > > Dear all, thanks for the hints. I was afraid I see ghosts. > > Yes, we have quotas on many directories. I'm not entirely sure if there were different quotas involved. I believe it was set only on the highest level, but now that you mention it ... > > I will try with quotas again, didn't include this in my scenario. > > This information is really important, because it will affect how I as an admin can move data between folders with quotas on them, which are typically used for sub-directory mounts. For a user this would look like a magical move between file systems. > > I will get back. Thanks! > > Best regards, > > ================= > Frank Schilder > AIT Risø Campus > Bygning 109, rum S14 > > ________________________________________ > From: Gregory Farnum <gfarnum@xxxxxxxxxx> > Sent: 26 March 2020 15:20:51 > To: Beocat KSU > Cc: ceph-users; Frank Schilder > Subject: Re: Re: Move on cephfs not O(1)? > > I was wondering that but at least in the userspace client the quotas > will just throw EXDEV or EDQUOT if it will exceed quotas... > ...and EXDEV might trigger the kernel to do a copy-and-delete, I > guess? Not sure. > > On Thu, Mar 26, 2020 at 7:18 AM Adam Tygart <mozes@xxxxxxx> wrote: > > > > Is there is a possibility that there was a quota involved? I've seen > > moves between quota zones to cause a copy then delete. > > > > -- > > Adam > > > > On Thu, Mar 26, 2020 at 9:14 AM Gregory Farnum <gfarnum@xxxxxxxxxx> wrote: > > > > > > On Thu, Mar 26, 2020 at 5:49 AM Frank Schilder <frans@xxxxxx> wrote: > > > > > > > > Some time ago I made a surprising observation. I reorganised a directory structure and needed to move a folder one level up with a command like > > > > > > > > mv A/B/ B > > > > > > > > B contained something like 9TB in very large files. To my surprise, this command didn't return for a couple of minutes and I started to look what was going on. What I discovered was, that the mv command actually performed a full copy with a subsequent remove. I had to wait for several hours for the move to complete. > > > > > > > > I tried to reproduce this today to collect further information. However, this behaviour seems not reproducible. No matter what I try, mv completes almost instantly. > > > > > > > > I was running the original mv on mimic 13.2.2 and retried now with mimic 13.2.8. In addition, there was an OS upgrade from Centos 7.6 to 7.7. I'm using the kernel-ml versions (5.xxx). Only one cephfs mount was present at all times. > > > > > > > > My questions are: > > > > > > > > 1) Was there a change from 13.2.2 to 13.2.8 explaining this? > > > > > > No. > > > > > > > 2) Are there (rare) conditions under which an mv on cephfs becomes a cp+rm? > > > > > > Not as part of CephFS. > > > > > > > 3) Am I seeing ghosts? > > > > > > The only way I can imagine this happening is if you had separate > > > CephFS mounts for cephfs/A/B and cephfs/ and you did the move between > > > them, instead of within the cephfs/ mount. :/ > > > -Greg > > > > > > > > > > > Thanks for clues and best regards, > > > > > > > > ================= > > > > Frank Schilder > > > > AIT Risø Campus > > > > Bygning 109, rum S14 > > > > _______________________________________________ > > > > ceph-users mailing list -- ceph-users@xxxxxxx > > > > To unsubscribe send an email to ceph-users-leave@xxxxxxx > > > > > > > _______________________________________________ > > > ceph-users mailing list -- ceph-users@xxxxxxx > > > To unsubscribe send an email to ceph-users-leave@xxxxxxx > > > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |