Thanks a lot! Have a good weekend. ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14 ________________________________________ From: Gregory Farnum <gfarnum@xxxxxxxxxx> Sent: 27 March 2020 21:18:37 To: Jeff Layton Cc: Frank Schilder; Zheng Yan; ceph-users; Luis Henriques Subject: Re: Re: Move on cephfs not O(1)? Made a ticket in the Linux Kernel Client tracker: https://tracker.ceph.com/issues/44791 I naively don't think this should be very complicated at all, except that I recall hearing something about locking issues with quota realms in the kclient? But the userspace client definitely doesn't have to do anything tricky with moving things, it's pretty much a normal move with some glue to point at a different quota realm. -Greg On Fri, Mar 27, 2020 at 7:34 AM Jeff Layton <jlayton@xxxxxxxxxx> wrote: > > On Fri, 2020-03-27 at 07:36 -0400, Jeff Layton wrote: > > On Thu, 2020-03-26 at 10:32 -0700, Gregory Farnum wrote: > > > On Thu, Mar 26, 2020 at 9:13 AM Frank Schilder <frans@xxxxxx> wrote: > > > > Dear all, > > > > > > > > yes, this is it, quotas. In the structure A/B/ there was a quota set on A. Hence, B was moved out of this zone and this does indeed change mv to be a cp+rm. > > > > > > > > The obvious follow-up. What is the procedure for properly moving data as an administrator? Do I really need to unset quotas, do the move and set quotas back again? > > > > > > Ah-ha, this is a difference between the userspace and kernel clients. > > > :( The kernel client always returns EXDEV if crossing "quota realms" > > > (different quota roots). I'm not sure why it behaves that way as > > > userspace is different: > > > * If fhere is a quota in the target directory, and > > > * If the target directory's existing data, plus the file(s) being > > > moved, exceed the target directory's quota, > > > then userspace returns EXDEV/EDQUOT. This seems like the right kernel > > > behavior as well... > > > > > > Jeff, is that a known issue for some reason? Should we make a new bug? :) > > > > > > > (cc'ing Luis since he wrote cafe21a4fb3075, which added this check) > > > > In ceph_rename, we have this, which is probably what you're hitting: > > > > /* don't allow cross-quota renames */ > > if ((old_dir != new_dir) && > > (!ceph_quota_is_same_realm(old_dir, new_dir))) > > return -EXDEV; > > > > That does seem to just check whether the realms are the same and doesn't > > actually check the space in them. > > > > I haven't studied this in detail, but it may be hard to ensure that we > > won't exceed the quota in a race-free way. How do you ensure that > > another thread doesn't do something that would exceed the quota just as > > you issue the rename request? > > > > This is fairly trivial in the userland client since it does everything > > under the BCM (Big Client Mutex), but won't be in the kernel client. > > > > Opening a bug for this won't hurt, but it may not be simple to > > implement. > > Ok, I gave it a harder look and it doesn't look too bad, given that > quotas are sort of "sloppy" anyway. We could just do a check that we > won't exceed quota on the destination directory. The catch is that there > may be other details to deal with too in moving an existing inode under > a different snaprealm. > > You're welcome to open a tracker bug for this, but I'm not sure who will > take on that project and when. > > Thanks, > -- > Jeff Layton <jlayton@xxxxxxxxxx> > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |