-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi,
I was setting up Ceph with CephFS over the past couple of days. Today,
I came to the point where I was able to mount CephFS and start playing
with accessing files.
Here is some output showing me performing some file operations on cephfs:
root@ceph4:/mnt/cephfs# ls -hl
total 223M
-rw-r--r-- 1 root root 222M Jan 10 15:43
debian-7.8.0-amd64-netinst.iso
-rw-rw-rw- 1 root root 6 Jan 21 12:45 testfile1
root@ceph4:/mnt/cephfs# cat testfile1
cat: testfile1: Operation not permitted
root@ceph4:/mnt/cephfs# cp debian-7.8.0-amd64-netinst.iso
debian-7.8.0-amd64-netinst.iso1
cp: reading `debian-7.8.0-amd64-netinst.iso': Operation not permitted
cp: failed to extend `debian-7.8.0-amd64-netinst.iso1': Operation
not permitted
root@ceph4:/mnt/cephfs# echo "blubb" > blubb
root@ceph4:/mnt/cephfs# cat blubb
blubb
root@ceph4:/mnt/cephfs# cat blubb
blubb
root@ceph4:/mnt/cephfs# cp blubb blurb
root@ceph4:/mnt/cephfs# cat blurb
blubb
root@ceph4:/mnt/cephfs# cat blurb
blubb
root@ceph4:/mnt/cephfs# rsync --progress -avz
debian-7.8.0-amd64-netinst.iso debian-7.8.0-amd64-netinst.iso1 > /dev/null
rsync: read errors mapping
"/mnt/cephfs/debian-7.8.0-amd64-netinst.iso": Operation not permitted (1)
WARNING: debian-7.8.0-amd64-netinst.iso failed verification --
update discarded (will try again).
rsync: read errors mapping
"/mnt/cephfs/debian-7.8.0-amd64-netinst.iso": Operation not permitted (1)
ERROR: debian-7.8.0-amd64-netinst.iso failed verification --
update discarded.
rsync error: some files/attrs were not transferred (see previous
errors) (code 23) at main.c(1070) [sender=3.0.9]
root@ceph4:/mnt/cephfs# cat blubb
blubb
root@ceph4:/mnt/cephfs# cat blubb
cat: blubb: Operation not permitted
root@ceph4:/mnt/cephfs#
You can see that after the failed attempt to copy the file with rsync,
I suddenly lost the ability to view file contents with cat. This is a
repeatable symptom. After restarting MDS, I am back at the point where
everything seems to work fine, until rsync destroys it. I believe I
tested with cp instead of rsync with the same results.
The closest thing to a hint of what is happening I could find was this:
root@ceph1:~# cat watch.sh
while [ 1 -eq 1 ]
do
date
ceph fs ls
sleep 5s
done
root@ceph1:~#
root@ceph1:~# bash watch.sh
Mi 21. Jan 15:34:05 CET 2015
No filesystems enabled
Mi 21. Jan 15:34:11 CET 2015
No filesystems enabled
Mi 21. Jan 15:34:16 CET 2015
name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ]
Mi 21. Jan 15:34:21 CET 2015
No filesystems enabled
Mi 21. Jan 15:34:26 CET 2015
No filesystems enabled
Mi 21. Jan 15:34:31 CET 2015
No filesystems enabled
Mi 21. Jan 15:34:36 CET 2015
No filesystems enabled
Mi 21. Jan 15:34:42 CET 2015
name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ]
Mi 21. Jan 15:34:47 CET 2015
No filesystems enabled
Mi 21. Jan 15:34:52 CET 2015
No filesystems enabled
Mi 21. Jan 15:34:57 CET 2015
No filesystems enabled
^C
root@ceph1:~#
You can see that "ceph fs ls" shows the filesystem sometimes and
sometimes not.
I could not see anything suspicuous in the logs, but that might be due
to lack of experience with Ceph logs. Therefore, I attached the logs
as a compressed tar.
Any help is highly appreciated!
cheers
Mathias
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQIcBAEBAgAGBQJUv8BIAAoJEA7RPiZyzgdI8TEP/3U50hrQuQNzBpmj2aJVkpxU
KFzEP/oyB2qlKlRobLVWRIXBFDH4K0auLvlAA9OwieGzS6+OoNKNO399Jopxku+w
DbR7eHAjgSOZDnkOyAtD12e/3J3Io3A+ImMpGheltOVYzwmAN2tXPaYkwTYhwS8t
i58KQvn3L254JtVS5kyTgHjlyQ5Kfo/lPcxLMVwPffy4O3cZ0y2YDG22NhvA38vm
u2EH5DgV/HCHrh1FOJAEmO3A+Ic2DSWG7vLPXbiww8UuTRijquaDYIlWBEFaSov/
zmMF71tax2P/BY3R0vOUE9Y99eHQbon7+L0WrPpjKdbih5KOtWS6zfKBYausNzW5
ryPL5eigQ0f2/GzF5BCUQsa1QvmY19HHRQ7n/b/OBPzNhEtbL5lQuqjPExrl9pZH
GhAYw+6kqywC4wJzdTzxIP/nhqFowcqLRD87GXGJBUBa8g88RAzpY3/rskKlKoBl
1Krx64kfud7CoJAKB0Nkbjk4dO5/PUC6hKTaZ5XYMIugXjRTeJxc61Cr+UmSVg8U
nsSouajvQlZ9at3VF8somn5N6zOHKNil9a9X+6nJcFeu0aY9MKNlUytWQemXuxi9
sj2ZMZZdXSDdkU2ovJVfq095AqDLPi8vO+RNx215pk7KaP79rEIjb+3btpV+TEA8
t3M9HgYv5gUs5qVSPtJQ
=PtqA
-----END PGP SIGNATURE-----
Attachment:
logs.tar.gz
Description: application/gzip
Attachment:
logs.tar.gz.sig
Description: PGP signature
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
