On Friday, February 8, 2013 at 8:42 AM, Yann Dupont wrote: > Le 08/02/2013 17:29, Gregory Farnum a écrit : > > > > > > On Friday, February 8, 2013, Yann Dupont wrote: > > > Le 08/02/2013 11:00, Tobias Prousa a écrit : > > > > Hi, > > > > > > > > I'm am evaluating a small ceph cluster with three nodes each running one mon, one mds, and 2-3 osds. Nodes are debian squeeze/wheezy (mixed) all running linux-3.2.35 from debian repos (from backports.debian.org (http://backports.debian.org) on squeeze box). Ceph is using repo from ceph.com/debian-bobtail (http://ceph.com/debian-bobtail), installed 0.56.2. > > > > > > > > The client is running debian wheezy but kernel 3.7.3 from debian experimental, ceph packages also installed using ceph.com/debian-bobtail (http://ceph.com/debian-bobtail) repo. > > > > > > > > There is only one active mds with two standby. One thing which might be unusual is that I do not mount the cephfs-root on the client but a subdir, i.e. mount -t ceph 172.16.0.4:6789:/home/ /home > > > > > > > > I am experiencing truncated files where files get writte successfully to ceph and can be read without problem, i.e. md5sum is ok and even filesize shows correct value of maybe 3.7MiB. Then after waiting for some time without the file receiving any IO it starts showing a filesize of 2.0MiB and md5sum fails (as to be expected). > > > > > > > > So far I only noticed that behaviour only in one subtree of my home folder where I prepare software packages using "tar cvfj ..." to create a .tar.bz2 bundle. Those files suddenly get truncated to 2.0MiB after some time, but it seems that only newly created files get truncated after some time. Similar .tar.bz2 files which I have there, since initial rsync to ceph, don't get truncated at all. > > > Hello, > > > I've seen the same, but because I used to point subdirectories to different pool (using cephs set_layout). > > > and restricted the client to writing on this very pool. > > > > > > caps: [osd] allow rw pool the-name-of-the-pool > > > > > > All seems to works ok, but after some times (or after an unmount/remount) all files are still there, but all content is nullified (yes,filled with 0). > > > > > > I ended up giving extend rights for the client ( caps: [osd] allow * ) and now it seems to works fine. > > > > > > BTW, is cephfs set_layout supposed to work ? > > > > > > If I set the layout for a subdirectory to, say pool 4, and restrict the client to another pool (say pool 5), I'm still able to mount, read and write the subdirectory. > > > > This'll happen because the client first writes to page cache (of course), and it doesn't discover it can't write out to the pool until it goes to flush. You should see errors if you try and fsync the files. (We have a bug open to add more pro-active checking of OSD write permissions.) Thanks Gregory for the explanation. That's more or less what I was imagining. So, the fact that file and their length are still there is because information is OK on mds, but no data is really written on osd, right ? > > Anyway, this explain the case for write, BUT why an unathorized client can still acces and read data from a pool he isn't allowed to do ? Still mds permissions ? Well, he shouldn't be able to read data off of the OSDs, but he'll still be able to see all the metadata which the MDS is making available to him (name, size, mtime, etc). _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |