Dear Ric, I played around a bit - the common denominator seems to be: Moving it within a directory subtree below a directory for which max_bytes / max_files quota settings are set, things work fine. Moving it to another directory tree without quota settings / with different quota settings, rename() returns EXDEV. Cheers, Oliver Am 25.05.2018 um 15:18 schrieb Ric Wheeler: > That seems to be the issue - we need to understand why rename sees them as different. > > Ric > > > On Fri, May 25, 2018, 9:15 AM Oliver Freyermuth <freyermuth@xxxxxxxxxxxxxxxxxx <mailto:freyermuth@xxxxxxxxxxxxxxxxxx>> wrote: > > Mhhhm... that's funny, I checked an mv with an strace now. I get: > --------------------------------------------------------------------------------- > access("/cephfs/some_folder/file", W_OK) = 0 > rename("foo", "/cephfs/some_folder/file") = -1 EXDEV (Invalid cross-device link) > unlink("/cephfs/some_folder/file") = 0 > lgetxattr("foo", "security.selinux", "system_u:object_r:fusefs_t:s0", 255) = 30 > --------------------------------------------------------------------------------- > But I can assure it's only a single filesystem, and a single ceph-fuse client running. > > Same happens when using absolute paths. > > Cheers, > Oliver > > Am 25.05.2018 um 15:06 schrieb Ric Wheeler: > > We should look at what mv uses to see if it thinks the directories are on different file systems. > > > > If the fstat or whatever it looks at is confused, that might explain it. > > > > Ric > > > > > > On Fri, May 25, 2018, 9:04 AM Oliver Freyermuth <freyermuth@xxxxxxxxxxxxxxxxxx <mailto:freyermuth@xxxxxxxxxxxxxxxxxx> <mailto:freyermuth@xxxxxxxxxxxxxxxxxx <mailto:freyermuth@xxxxxxxxxxxxxxxxxx>>> wrote: > > > > Am 25.05.2018 um 14:57 schrieb Ric Wheeler: > > > Is this move between directories on the same file system? > > > > It is, we only have a single CephFS in use. There's also only a single ceph-fuse client running. > > > > What's different, though, are different ACLs set for source and target directory, and owner / group, > > but I hope that should not matter. > > > > All the best, > > Oliver > > > > > Rename as a system call only works within a file system. > > > > > > The user space mv command becomes a copy when not the same file system. > > > > > > Regards, > > > > > > Ric > > > > > > > > > On Fri, May 25, 2018, 8:51 AM John Spray <jspray@xxxxxxxxxx <mailto:jspray@xxxxxxxxxx> <mailto:jspray@xxxxxxxxxx <mailto:jspray@xxxxxxxxxx>> <mailto:jspray@xxxxxxxxxx <mailto:jspray@xxxxxxxxxx> <mailto:jspray@xxxxxxxxxx <mailto:jspray@xxxxxxxxxx>>>> wrote: > > > > > > On Fri, May 25, 2018 at 1:10 PM, Oliver Freyermuth > > > <freyermuth@xxxxxxxxxxxxxxxxxx <mailto:freyermuth@xxxxxxxxxxxxxxxxxx> <mailto:freyermuth@xxxxxxxxxxxxxxxxxx <mailto:freyermuth@xxxxxxxxxxxxxxxxxx>> <mailto:freyermuth@xxxxxxxxxxxxxxxxxx <mailto:freyermuth@xxxxxxxxxxxxxxxxxx> <mailto:freyermuth@xxxxxxxxxxxxxxxxxx <mailto:freyermuth@xxxxxxxxxxxxxxxxxx>>>> wrote: > > > > Dear Cephalopodians, > > > > > > > > I was wondering why a simple "mv" is taking extraordinarily long on CephFS and must note that, > > > > at least with the fuse-client (12.2.5) and when moving a file from one directory to another, > > > > the file appears to be copied first (byte by byte, traffic going through the client?) before the initial file is deleted. > > > > > > > > Is this true, or am I missing something? > > > > > > A mv should not involve copying a file through the client -- it's > > > implemented in the MDS as a rename from one location to another. > > > What's the observation that's making it seem like the data is going > > > through the client? > > > > > > John > > > > > > > > > > > For large files, this might be rather time consuming, > > > > and we should certainly advise all our users to not move files around needlessly if this is the case. > > > > > > > > Cheers, > > > > Oliver > > > > > > > > > > > > _______________________________________________ > > > > ceph-users mailing list > > > > ceph-users@xxxxxxxxxxxxxx <mailto:ceph-users@xxxxxxxxxxxxxx> <mailto:ceph-users@xxxxxxxxxxxxxx <mailto:ceph-users@xxxxxxxxxxxxxx>> <mailto:ceph-users@xxxxxxxxxxxxxx <mailto:ceph-users@xxxxxxxxxxxxxx> <mailto:ceph-users@xxxxxxxxxxxxxx <mailto:ceph-users@xxxxxxxxxxxxxx>>> > > > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > > > > _______________________________________________ > > > ceph-users mailing list > > > ceph-users@xxxxxxxxxxxxxx <mailto:ceph-users@xxxxxxxxxxxxxx> <mailto:ceph-users@xxxxxxxxxxxxxx <mailto:ceph-users@xxxxxxxxxxxxxx>> <mailto:ceph-users@xxxxxxxxxxxxxx <mailto:ceph-users@xxxxxxxxxxxxxx> <mailto:ceph-users@xxxxxxxxxxxxxx <mailto:ceph-users@xxxxxxxxxxxxxx>>> > > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > > > > >
Attachment:
smime.p7s
Description: S/MIME Cryptographic Signature
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
