Re: CephFS "move" operation

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Oliver Freyermuth <freyermuth@xxxxxxxxxxxxxxxxxx> writes:

> Mhhhm... that's funny, I checked an mv with an strace now. I get:
> ---------------------------------------------------------------------------------
> access("/cephfs/some_folder/file", W_OK) = 0
> rename("foo", "/cephfs/some_folder/file") = -1 EXDEV (Invalid cross-device link)

I believe this could happen if you have quotas set on any of the paths,
or different snapshot realms.

Cheers,
-- 
Luis


> unlink("/cephfs/some_folder/file") = 0 lgetxattr("foo",
> "security.selinux", "system_u:object_r:fusefs_t:s0", 255) = 30
> ---------------------------------------------------------------------------------
> But I can assure it's only a single filesystem, and a single ceph-fuse
> client running.
>
> Same happens when using absolute paths. 
>
> Cheers,
> 	Oliver
>
> Am 25.05.2018 um 15:06 schrieb Ric Wheeler:
>> We should look at what mv uses to see if it thinks the directories are on different file systems.
>> 
>> If the fstat or whatever it looks at is confused, that might explain it.
>> 
>> Ric
>> 
>> 
>> On Fri, May 25, 2018, 9:04 AM Oliver Freyermuth <freyermuth@xxxxxxxxxxxxxxxxxx <mailto:freyermuth@xxxxxxxxxxxxxxxxxx>> wrote:
>> 
>>     Am 25.05.2018 um 14:57 schrieb Ric Wheeler:
>>     > Is this move between directories on the same file system?
>> 
>>     It is, we only have a single CephFS in use. There's also only a single ceph-fuse client running.
>> 
>>     What's different, though, are different ACLs set for source and target directory, and owner / group,
>>     but I hope that should not matter.
>> 
>>     All the best,
>>     Oliver
>> 
>>     > Rename as a system call only works within a file system.
>>     >
>>     > The user space mv command becomes a copy when not the same file system. 
>>     >
>>     > Regards,
>>     >
>>     > Ric
>>     >
>>     >
>>     > On Fri, May 25, 2018, 8:51 AM John Spray <jspray@xxxxxxxxxx <mailto:jspray@xxxxxxxxxx> <mailto:jspray@xxxxxxxxxx <mailto:jspray@xxxxxxxxxx>>> wrote:
>>     >
>>     >     On Fri, May 25, 2018 at 1:10 PM, Oliver Freyermuth
>>     >     <freyermuth@xxxxxxxxxxxxxxxxxx <mailto:freyermuth@xxxxxxxxxxxxxxxxxx> <mailto:freyermuth@xxxxxxxxxxxxxxxxxx <mailto:freyermuth@xxxxxxxxxxxxxxxxxx>>> wrote:
>>     >     > Dear Cephalopodians,
>>     >     >
>>     >     > I was wondering why a simple "mv" is taking extraordinarily long on CephFS and must note that,
>>     >     > at least with the fuse-client (12.2.5) and when moving a file from one directory to another,
>>     >     > the file appears to be copied first (byte by byte, traffic going through the client?) before the initial file is deleted.
>>     >     >
>>     >     > Is this true, or am I missing something?
>>     >
>>     >     A mv should not involve copying a file through the client -- it's
>>     >     implemented in the MDS as a rename from one location to another.
>>     >     What's the observation that's making it seem like the data is going
>>     >     through the client?
>>     >
>>     >     John
>>     >
>>     >     >
>>     >     > For large files, this might be rather time consuming,
>>     >     > and we should certainly advise all our users to not move files around needlessly if this is the case.
>>     >     >
>>     >     > Cheers,
>>     >     >         Oliver
>>     >     >
>>     >     >
>>     >     > _______________________________________________
>>     >     > ceph-users mailing list
>>     >     > ceph-users@xxxxxxxxxxxxxx <mailto:ceph-users@xxxxxxxxxxxxxx> <mailto:ceph-users@xxxxxxxxxxxxxx <mailto:ceph-users@xxxxxxxxxxxxxx>>
>>     >     > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>     >     >
>>     >     _______________________________________________
>>     >     ceph-users mailing list
>>     >     ceph-users@xxxxxxxxxxxxxx <mailto:ceph-users@xxxxxxxxxxxxxx> <mailto:ceph-users@xxxxxxxxxxxxxx <mailto:ceph-users@xxxxxxxxxxxxxx>>
>>     >     http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>     >
>> 
>
>
>
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux