howdy, y'all.
--
we are testing ceph and all of its features. we love RBD! however cephFS, though clearly stated not production ready, has been stonewalling us. in an attempt to get rolling quickly, we followed some guides on CephFS (http://goo.gl/BmVxG, http://goo.gl/1VtNk).
when i mount CephFS, i can:
1) create a file with contents on clientA;
2) list the folder and see the file on clientB and clientC;
3) pull up the exact same xattr's on that file across all clients.
however, as soon as i try to modify or view the contents of the file, ubuntu 12.04 says "operation not permitted". only clientA, which made the file, can try to view it's contents, but when clientA does so the file appears empty. a remount of the share on any client makes no difference. for what it is worth, xattrs don't change upon remount.
what in the world are we doing wrong?
cephx is enabled. works great for RBD. our ceph auth command for cephFS user:
ceph auth get-or-create client.cephfs mds 'allow *' mon 'allow r' osd 'allow rwx pool=cephfs'
we are allowing ceph to determine what pool to use, which i think defaults to 0/"data".
mount command:
mount -t ceph 10.0.10.53:6789:/ /mnt/test/ -o name=cephfs,secretfile=/etc/ceph/client.cephfs,noatime
mount path exists; permissions are consistent across all clients. "/etc/ceph/client.cephfs" is populated with the output of "ceph auth print-key client.cephfs".
all my clients are currently ubuntu 12.04 LTS within virtual machines. our company has not dedicated physical servers to this project.
kernel for all clients:
Linux os-comp2 3.5.0-30-generic #51~precise1-Ubuntu SMP Wed May 15 08:48:19 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
xattrs for test file (removed the hex code):
# xattr -l testfile
ceph.file.layout:
chunk_bytes=4194304.stripe_count=1.object_size=4194304.
ceph.layout:
chunk_bytes=4194304.stripe_count=1.object_size=4194304.
validating ceph actually created the file, in lue of http://tracker.ceph.com/issues/2753 :
# cephfs testfile show_location
location.file_offset: 0
location.object_offset:0
location.object_no: 0
location.object_size: 4194304
location.object_name: 10000000000.00000000
location.block_offset: 0
location.block_size: 4194304
location.osd: 4
-bo
"But God demonstrates His own love toward us, in that while we were yet sinners, Christ died for us. Much more then, having now been justified by His blood, we shall be saved from the wrath of God through Him." Romans 5:8-9
All have sinned, broken God's law, and deserve eternal torment. Jesus Christ, the Son of God, died for the sins of those that will believe, purchasing our salvation, and defeated death so that we all may spend eternity in heaven. Do you desire freedom from hell and be with God in His love for eternity?
"If you confess with your mouth Jesus as Lord, and believe in your heart that God raised Him from the dead, you will be saved." Romans 10:9
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
