Thanks John, Greg. If I understand this correctly, then, doing this: rados -p hotpool cache-flush-evict-all should start appropriately deleting objects from the cache pool. I just started one up, and that seems to be working. Otherwise, the cache's confgured timeouts/limits should get those deletions propagated through to the cold pool naturally. Is that right? Thanks again, Lincoln On Jun 12, 2015, at 1:12 PM, Gregory Farnum wrote: > On Fri, Jun 12, 2015 at 11:07 AM, John Spray <john.spray@xxxxxxxxxx> wrote: >> >> Just had a go at reproducing this, and yeah, the behaviour is weird. Our >> automated testing for cephfs doesn't include any cache tiering, so this is a >> useful exercise! >> >> With a writeback overlay cache tier pool on an EC pool, I write a bunch of >> files, then do a rados cache-flush-evict-all, then delete the files in >> cephfs. The result is that all the objects are still present in a "rados >> ls" on either base or cache pool, but if I try to rm any of them I get an >> ENOENT. >> >> Then, finally, when I do another cache-flush-evict-all, now the objects are >> all finally disappearing from the df stats (base and cache pool stats >> ticking down together). >> >> So intuitively, I guess the cache tier is caching the delete-ness of the >> objects, and only later flushing that (i.e. deleting from the base pool). >> The object is still "in the cache" on that basis, and presumably not getting >> flushed (i.e. deleting in base pool) until usual timeouts/space limits >> apply. > > Yep, that's it exactly. This is expected behavior. > >> Maybe we need something to kick delete flushes to happen much >> earlier (like, ASAP when the cluster isn't too busy doing other >> promotions/evictions). > > Sounds like a good RADOS feature request/blueprint that somebody in > the community might be able to handle. > -Greg _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |