Thanks for the correction on IRC. I should have written that this issue
started with 0.59 (when the monitor changes hit).
http://ceph.com/dev-notes/cephs-new-monitor-changes/
The writeup and release notes sometimes say they went in for 0.58, but I
believe they were actually released in 0.59.
Thanks for the correction Sylvain.
- Mike
On 5/21/2013 8:57 AM, Mike Dawson wrote:
Sylvain,
I can confirm I see a similar traffic pattern.
Any time I have lots of writes going to my cluster (like heavy writes
from RBD or remapping/backfilling after losing an OSD), I see all sorts
of monitor issues.
If my monitor leveldb store.db directories grow past some unknown point
(maybe ~1GB or so), 'compact on trim' is insufficiently slow. The
store.db grows faster than compact can trim the garbage. After that
point, the only hope to rein in the store.db size is to stop the OSDs
and get leveldb to compact without any ongoing writes.
I sent Sage and Joao a transaction dump of the growth yesterday. Sage
looked, but the files are so large it is tough to get useful info.
http://tracker.ceph.com/issues/4895
I believe this issue has existed since 0.48.
- Mike
On 5/21/2013 8:16 AM, Sylvain Munaut wrote:
Hi,
I've just added some monitoring to the IO usage of mon (trying to
track down that growing mon issue), and I'm kind of surprised by the
amount of IO generated by the monitor process.
I get continuous 4 Mo/s / 75 iops with added big spikes at each
compaction every 3 min or so.
Is there a description somewhere of what the monitor does exactly ? I
mean the monmap / pgmap / osdmap / mdsmap / election epoch don't
change that often (pgmap is like 1 per second and that's the fastest
change by several orders of magnitude). So what exactly does the
monitor do with all that IO ???
Cheers,
Sylvain
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
