Dear Sage,
here you go, some_folder in reality is "/cephfs/group":
------------------------------------------------
# stat foo
File: ‘foo’
Size: 1048576000 Blocks: 2048000 IO Block: 4194304 regular file
Device: 27h/39d Inode: 1099515065517 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Context: system_u:object_r:fusefs_t:s0
Access: 2018-05-25 15:27:59.433279424 +0200
Modify: 2018-05-25 15:28:01.379754052 +0200
Change: 2018-05-25 15:28:01.379754052 +0200
Birth: -
------------------------------------------------
# stat -f foo
File: "foo"
ID: 0 Namelen: 255 Type: fuseblk
Block size: 4194304 Fundamental block size: 4194304
Blocks: Total: 104471885 Free: 79096968 Available: 79096968
Inodes: Total: 26258533 Free: -1
------------------------------------------------
------------------------------------------------
# stat -f /cephfs/group/
File: "/cephfs/group/"
ID: 0 Namelen: 255 Type: fuseblk
Block size: 4194304 Fundamental block size: 4194304
Blocks: Total: 104471835 Free: 79098264 Available: 79098264
Inodes: Total: 26257190 Free: -1
------------------------------------------------
# stat /cephfs/group/
File: ‘/cephfs/group/’
Size: 73167320986856 Blocks: 1 IO Block: 4096 directory
Device: 27h/39d Inode: 1099511627888 Links: 1
Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
Context: system_u:object_r:fusefs_t:s0
Access: 2018-03-09 18:22:47.061501906 +0100
Modify: 2018-05-25 15:18:02.164391701 +0200
Change: 2018-05-25 15:18:02.164391701 +0200
Birth: -
------------------------------------------------
Cheers,
Oliver
Am 25.05.2018 um 15:21 schrieb Sage Weil:
> Can you paste the output of 'stat foo' and 'stat /cephfs/some_folder'?
> (Maybe also the same with 'stat -f'.)
>
> Thanks!
> sage
>
>
> On Fri, 25 May 2018, Ric Wheeler wrote:
>> That seems to be the issue - we need to understand why rename sees them as
>> different.
>>
>> Ric
>>
>>
>> On Fri, May 25, 2018, 9:15 AM Oliver Freyermuth <
>> freyermuth@xxxxxxxxxxxxxxxxxx> wrote:
>>
>>> Mhhhm... that's funny, I checked an mv with an strace now. I get:
>>>
>>> ---------------------------------------------------------------------------------
>>> access("/cephfs/some_folder/file", W_OK) = 0
>>> rename("foo", "/cephfs/some_folder/file") = -1 EXDEV (Invalid cross-device
>>> link)
>>> unlink("/cephfs/some_folder/file") = 0
>>> lgetxattr("foo", "security.selinux", "system_u:object_r:fusefs_t:s0", 255)
>>> = 30
>>>
>>> ---------------------------------------------------------------------------------
>>> But I can assure it's only a single filesystem, and a single ceph-fuse
>>> client running.
>>>
>>> Same happens when using absolute paths.
>>>
>>> Cheers,
>>> Oliver
>>>
>>> Am 25.05.2018 um 15:06 schrieb Ric Wheeler:
>>>> We should look at what mv uses to see if it thinks the directories are
>>> on different file systems.
>>>>
>>>> If the fstat or whatever it looks at is confused, that might explain it.
>>>>
>>>> Ric
>>>>
>>>>
>>>> On Fri, May 25, 2018, 9:04 AM Oliver Freyermuth <
>>> freyermuth@xxxxxxxxxxxxxxxxxx <mailto:freyermuth@xxxxxxxxxxxxxxxxxx>>
>>> wrote:
>>>>
>>>> Am 25.05.2018 um 14:57 schrieb Ric Wheeler:
>>>> > Is this move between directories on the same file system?
>>>>
>>>> It is, we only have a single CephFS in use. There's also only a
>>> single ceph-fuse client running.
>>>>
>>>> What's different, though, are different ACLs set for source and
>>> target directory, and owner / group,
>>>> but I hope that should not matter.
>>>>
>>>> All the best,
>>>> Oliver
>>>>
>>>> > Rename as a system call only works within a file system.
>>>> >
>>>> > The user space mv command becomes a copy when not the same file
>>> system.
>>>> >
>>>> > Regards,
>>>> >
>>>> > Ric
>>>> >
>>>> >
>>>> > On Fri, May 25, 2018, 8:51 AM John Spray <jspray@xxxxxxxxxx
>>> <mailto:jspray@xxxxxxxxxx> <mailto:jspray@xxxxxxxxxx <mailto:
>>> jspray@xxxxxxxxxx>>> wrote:
>>>> >
>>>> > On Fri, May 25, 2018 at 1:10 PM, Oliver Freyermuth
>>>> > <freyermuth@xxxxxxxxxxxxxxxxxx <mailto:
>>> freyermuth@xxxxxxxxxxxxxxxxxx> <mailto:freyermuth@xxxxxxxxxxxxxxxxxx
>>> <mailto:freyermuth@xxxxxxxxxxxxxxxxxx>>> wrote:
>>>> > > Dear Cephalopodians,
>>>> > >
>>>> > > I was wondering why a simple "mv" is taking extraordinarily
>>> long on CephFS and must note that,
>>>> > > at least with the fuse-client (12.2.5) and when moving a
>>> file from one directory to another,
>>>> > > the file appears to be copied first (byte by byte, traffic
>>> going through the client?) before the initial file is deleted.
>>>> > >
>>>> > > Is this true, or am I missing something?
>>>> >
>>>> > A mv should not involve copying a file through the client --
>>> it's
>>>> > implemented in the MDS as a rename from one location to
>>> another.
>>>> > What's the observation that's making it seem like the data is
>>> going
>>>> > through the client?
>>>> >
>>>> > John
>>>> >
>>>> > >
>>>> > > For large files, this might be rather time consuming,
>>>> > > and we should certainly advise all our users to not move
>>> files around needlessly if this is the case.
>>>> > >
>>>> > > Cheers,
>>>> > > Oliver
>>>> > >
>>>> > >
>>>> > > _______________________________________________
>>>> > > ceph-users mailing list
>>>> > > ceph-users@xxxxxxxxxxxxxx <mailto:ceph-users@xxxxxxxxxxxxxx>
>>> <mailto:ceph-users@xxxxxxxxxxxxxx <mailto:ceph-users@xxxxxxxxxxxxxx>>
>>>> > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>>> > >
>>>> > _______________________________________________
>>>> > ceph-users mailing list
>>>> > ceph-users@xxxxxxxxxxxxxx <mailto:ceph-users@xxxxxxxxxxxxxx>
>>> <mailto:ceph-users@xxxxxxxxxxxxxx <mailto:ceph-users@xxxxxxxxxxxxxx>>
>>>> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>>> >
>>>>
>>>
>>>
>>>
>>>
>>
--
Oliver Freyermuth
Universität Bonn
Physikalisches Institut, Raum 1.047
Nußallee 12
53115 Bonn
--
Tel.: +49 228 73 2367
Fax: +49 228 73 7869
--
Attachment:
smime.p7s
Description: S/MIME Cryptographic Signature
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
