Hi, > We actually saw a decrease in memory usage after upgrading to Firefly, > though we did reboot the nodes after the upgrade while we had the > maintenance window. This is with 216 OSDs total (32-40 per node): > http://i.imgur.com/BC7RuXJ.png Interesting. Is that cluster for RBD or RGW ? My RBD OSDs are a bit better behaved but still had this 25% bump in mem usage ... Here the memory pretty much just grows continually. This is the log over the last year. http://i.imgur.com/0NUFjpz.png At the very beginning (~250M per process) those OSD were empty, just added. Then we changed the crushmap to map all the RGW pools we have to them, then it just grows slowly with a bump at pretty much each update. And this is a pretty small pool of OSDs, for theses there is only 8 OSD processes over 4 nodes, storing barely 1 To in 2.5 millions objects, split into 7 pools and 5376 PGs (some pools have size=3, other size=2) 1.5 Go per OSD process seems a bit big to me. Cheers, Sylvain
 |