On Thu, Sep 11, 2014 at 11:39 AM, Sage Weil <sweil at redhat.com> wrote: > On Thu, 11 Sep 2014, Gregory Farnum wrote: >> On Thu, Sep 11, 2014 at 4:13 AM, Kenneth Waegeman >> <Kenneth.Waegeman at ugent.be> wrote: >> > Hi all, >> > >> > I am testing the tiering functionality with cephfs. I used a replicated >> > cache with an EC data pool, and a replicated metadata pool like this: >> > >> > >> > ceph osd pool create cache 1024 1024 >> > ceph osd pool set cache size 2 >> > ceph osd pool set cache min_size 1 >> > ceph osd erasure-code-profile set profile11 k=8 m=3 >> > ruleset-failure-domain=osd >> > ceph osd pool create ecdata 128 128 erasure profile11 >> > ceph osd tier add ecdata cache >> > ceph osd tier cache-mode cache writeback >> > ceph osd tier set-overlay ecdata cache >> > ceph osd pool set cache hit_set_type bloom >> > ceph osd pool set cache hit_set_count 1 >> > ceph osd pool set cache hit_set_period 3600 >> > ceph osd pool set cache target_max_bytes $((280*1024*1024*1024)) >> > ceph osd pool create metadata 128 128 >> > ceph osd pool set metadata crush_ruleset 1 # SSD root in crushmap >> > ceph fs new ceph_fs metadata cache <-- wrong ? >> > >> > I started testing with this, and this worked, I could write to it with >> > cephfs and the cache was flushing to the ecdata pool as expected. >> > But now I notice I made the fs right upon the cache, instead of the >> > underlying data pool. I suppose I should have done this: >> > >> > ceph fs new ceph_fs metadata ecdata >> > >> > So my question is: Was this wrong and not doing the things I thought it did, >> > or was this somehow handled by ceph and didn't it matter I specified the >> > cache instead of the data pool? >> >> Well, it's sort of doing what you want it to. You've told the >> filesystem to use the "cache" pool as the location for all of its >> data. But RADOS is pushing everything in the "cache" pool down to the >> "ecdata" pool. >> So it'll work for now as you want. But if in future you wanted to stop >> using the caching pool, or switch it out for a different pool >> entirely, that wouldn't work (whereas it would if the fs was using >> "ecdata"). >> >> We should perhaps look at prevent use of cache pools like this...hrm... >> http://tracker.ceph.com/issues/9435 > > Should we? I was planning on doing exactly this for my home cluster. Not cache pools under CephFS, but specifying the cache pool as the data pool (rather than some underlying pool). Or is there some reason we might want the cache pool to be the one the filesystem is using for indexing? -Greg
 |