On Sun, 6 Mar 2016 12:17:48 +0300 Mike Almateia wrote: > Hello Cephers! > > When my cluster hit "full ratio" settings, objects from cache pull > didn't flush to a cold storage. > As always, versions of everything, Ceph foremost. > 1. Hit the 'full ratio': > > 2016-03-06 11:35:23.838401 osd.64 10.22.11.21:6824/31423 4327 : cluster > [WRN] OSD near full (90%) > 2016-03-06 11:35:55.447205 osd.64 10.22.11.21:6824/31423 4329 : cluster > [WRN] OSD near full (90%) > 2016-03-06 11:36:29.255815 osd.64 10.22.11.21:6824/31423 4332 : cluster > [WRN] OSD near full (90%) > 2016-03-06 11:37:04.769765 osd.64 10.22.11.21:6824/31423 4333 : cluster > [WRN] OSD near full (90%) > ... > You want to: a) read the latest (master) documentation for cache tiering b) this ML and it archives, in particular the current thread titled "Cache tier operation clarifications" In short, target_max_bytes or objects NEEDs to be set. > 2. Well, ok. Set the option 'ceph osd pool set hotec > cache_target_full_ratio 0.8'. > But no one of objects didn't flush at all > Flush and evict are 2 different things. cache_target_dirty_ratio needs to be set as well (below full) for this to work, aside from the issue above. > 3. Ok. Try flush all object manually: > [root@c1 ~]# rados -p hotec cache-flush-evict-all > > rbd_data.34d1f5746d773.0000000000016ba9 > > 4. After full day objects still in cache pool, didn't flush at all: > [root@c1 ~]# rados df > pool name KB objects clones degraded > unfound rd rd KB wr wr KB > data 0 0 0 0 > 0 6 4 158212 215700473 > hotec 797656118 25030755 0 0 > 0 370599 163045649 69947951 17786794779 > rbd 0 0 0 0 > 0 0 0 0 0 > total used 2080570792 25030755 > > It a bug or predictable action? > If you didn't set the cache to forward mode first, it will fill up again immediately. Christian -- Christian Balzer Network/Systems Engineer chibi@xxxxxxx Global OnLine Japan/Rakuten Communications http://www.gol.com/ _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |