Holy cow.
Thank you for pointing out what should have been obvious. So glad these emails are kept on the web for future searchers like me ;)
-bo
On Tue, Jun 11, 2013 at 11:46 AM, Gregory Farnum <greg@xxxxxxxxxxx> wrote:
On Tue, Jun 11, 2013 at 9:39 AM, Bo <bo@xxxxxxxxxxx> wrote:Yep, that's why. Your cephx key only lets the client touch the
> howdy, y'all.
>
> we are testing ceph and all of its features. we love RBD! however cephFS,
> though clearly stated not production ready, has been stonewalling us. in an
> attempt to get rolling quickly, we followed some guides on CephFS
> (http://goo.gl/BmVxG, http://goo.gl/1VtNk).
>
> when i mount CephFS, i can:
> 1) create a file with contents on clientA;
> 2) list the folder and see the file on clientB and clientC;
> 3) pull up the exact same xattr's on that file across all clients.
>
> however, as soon as i try to modify or view the contents of the file, ubuntu
> 12.04 says "operation not permitted". only clientA, which made the file, can
> try to view it's contents, but when clientA does so the file appears empty.
> a remount of the share on any client makes no difference. for what it is
> worth, xattrs don't change upon remount.
>
> what in the world are we doing wrong?
>
> cephx is enabled. works great for RBD. our ceph auth command for cephFS
> user:
> ceph auth get-or-create client.cephfs mds 'allow *' mon 'allow r' osd 'allow
> rwx pool=cephfs'
>
> we are allowing ceph to determine what pool to use, which i think defaults
> to 0/"data".
"cephfs" pool, but the metadata server is telling the client to put
its file data in the "data" pool. The client buffers up writes, sends
all the metadata to the server (where it is successfully stored), but
when it tries to flush anything out it fails to do so. You'll need to
resolve that discrepancy, either by reconfiguring CephFS to store all
the file data in the "cephfs" pool (look up the newfs command in the
docs, or just set a new layout on the root dir), or adding the "data"
pool to the list that the clients can access.
-Greg
Software Engineer #42 @ http://inktank.com | http://ceph.com
"But God demonstrates His own love toward us, in that while we were yet sinners, Christ died for us. Much more then, having now been justified by His blood, we shall be saved from the wrath of God through Him." Romans 5:8-9
All have sinned, broken God's law, and deserve eternal torment. Jesus Christ, the Son of God, died for the sins of those that will believe, purchasing our salvation, and defeated death so that we all may spend eternity in heaven. Do you desire freedom from hell and be with God in His love for eternity?
"If you confess with your mouth Jesus as Lord, and believe in your heart that God raised Him from the dead, you will be saved." Romans 10:9
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
