Re: Move on cephfs not O(1)?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Made a ticket in the Linux Kernel Client tracker:
https://tracker.ceph.com/issues/44791

I naively don't think this should be very complicated at all, except
that I recall hearing something about locking issues with quota realms
in the kclient? But the userspace client definitely doesn't have to do
anything tricky with moving things, it's pretty much a normal move
with some glue to point at a different quota realm.
-Greg

On Fri, Mar 27, 2020 at 7:34 AM Jeff Layton <jlayton@xxxxxxxxxx> wrote:
>
> On Fri, 2020-03-27 at 07:36 -0400, Jeff Layton wrote:
> > On Thu, 2020-03-26 at 10:32 -0700, Gregory Farnum wrote:
> > > On Thu, Mar 26, 2020 at 9:13 AM Frank Schilder <frans@xxxxxx> wrote:
> > > > Dear all,
> > > >
> > > > yes, this is it, quotas. In the structure A/B/ there was a quota set on A. Hence, B was moved out of this zone and this does indeed change mv to be a cp+rm.
> > > >
> > > > The obvious follow-up. What is the procedure for properly moving data as an administrator? Do I really need to unset quotas, do the move and set quotas back again?
> > >
> > > Ah-ha, this is a difference between the userspace and kernel clients.
> > > :( The kernel client always returns EXDEV if crossing "quota realms"
> > > (different quota roots). I'm not sure why it behaves that way as
> > > userspace is different:
> > > * If fhere is a quota in the target directory, and
> > > * If the target directory's existing data, plus the file(s) being
> > > moved, exceed the target directory's quota,
> > > then userspace returns EXDEV/EDQUOT. This seems like the right kernel
> > > behavior as well...
> > >
> > > Jeff, is that a known issue for some reason? Should we make a new bug? :)
> > >
> >
> > (cc'ing Luis since he wrote cafe21a4fb3075, which added this check)
> >
> > In ceph_rename, we have this, which is probably what you're hitting:
> >
> >         /* don't allow cross-quota renames */
> >         if ((old_dir != new_dir) &&
> >             (!ceph_quota_is_same_realm(old_dir, new_dir)))
> >                 return -EXDEV;
> >
> > That does seem to just check whether the realms are the same and doesn't
> > actually check the space in them.
> >
> > I haven't studied this in detail, but it may be hard to ensure that we
> > won't exceed the quota in a race-free way. How do you ensure that
> > another thread doesn't do something that would exceed the quota just as
> > you issue the rename request?
> >
> > This is fairly trivial in the userland client since it does everything
> > under the BCM (Big Client Mutex), but won't be in the kernel client.
> >
> > Opening a bug for this won't hurt, but it may not be simple to
> > implement.
>
> Ok, I gave it a harder look and it doesn't look too bad, given that
> quotas are sort of "sloppy" anyway. We could just do a check that we
> won't exceed quota on the destination directory. The catch is that there
> may be other details to deal with too in moving an existing inode under
> a different snaprealm.
>
> You're welcome to open a tracker bug for this, but I'm not sure who will
> take on that project and when.
>
> Thanks,
> --
> Jeff Layton <jlayton@xxxxxxxxxx>
>
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux