On Tue, Jun 11, 2013 at 9:39 AM, Bo <bo@xxxxxxxxxxx> wrote: > howdy, y'all. > > we are testing ceph and all of its features. we love RBD! however cephFS, > though clearly stated not production ready, has been stonewalling us. in an > attempt to get rolling quickly, we followed some guides on CephFS > (http://goo.gl/BmVxG, http://goo.gl/1VtNk). > > when i mount CephFS, i can: > 1) create a file with contents on clientA; > 2) list the folder and see the file on clientB and clientC; > 3) pull up the exact same xattr's on that file across all clients. > > however, as soon as i try to modify or view the contents of the file, ubuntu > 12.04 says "operation not permitted". only clientA, which made the file, can > try to view it's contents, but when clientA does so the file appears empty. > a remount of the share on any client makes no difference. for what it is > worth, xattrs don't change upon remount. > > what in the world are we doing wrong? > > cephx is enabled. works great for RBD. our ceph auth command for cephFS > user: > ceph auth get-or-create client.cephfs mds 'allow *' mon 'allow r' osd 'allow > rwx pool=cephfs' > > we are allowing ceph to determine what pool to use, which i think defaults > to 0/"data". Yep, that's why. Your cephx key only lets the client touch the "cephfs" pool, but the metadata server is telling the client to put its file data in the "data" pool. The client buffers up writes, sends all the metadata to the server (where it is successfully stored), but when it tries to flush anything out it fails to do so. You'll need to resolve that discrepancy, either by reconfiguring CephFS to store all the file data in the "cephfs" pool (look up the newfs command in the docs, or just set a new layout on the root dir), or adding the "data" pool to the list that the clients can access. -Greg Software Engineer #42 @ http://inktank.com | http://ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |