Thanks Sage, the cache system's look pretty great so far. Combined with erasure coding it's really adding a lot of options. -Michael On 21/05/2014 21:54, Sage Weil wrote: > On Wed, 21 May 2014, Michael wrote: >> Hi All, >> >> Experimenting with cache pools for RBD, created two pools, slowdata-hot backed >> by slowdata-cold. Set up max data to be stored in hot to be 100GB, data to be >> moved to cold above 40% hot usage. Created a 100GB RBD image, mounted it >> tested reading/writing, then dumped in 80GB of data. All looked to be going >> well. Left everything to settle down and then unmounted the RBD and deleted >> it. Afterwards: >> >> pool name category KB objects clones degraded >> unfound rd rd KB wr wr KB >> slowdata-cold - 55835980 13817 0 0 >> 0 27741 3 16818 67715123 >> slowdata-hot - 12 23610 0 0 >> 0 1863785 112250386 150951 138481712 >> >> Deleting the RBD image cleaned out the hot pool's data but left the cold >> pool full of data and both of them full of objects. Anyone else trying >> this out? > This is normal. The cache pool contains a sort of "whiteout" that > indicates the object is deleted, but it isn't cold enough yet to have > flushed that change to the base pool. As you write more data you will > find the backend pool's objects for that image will eventually disappear. > > sage