Hi Andrej, That doesn't sound right -- I checked a couple of our clusters just now and the mon filesystem is writing at just a few 100kBps. debug_mon = 10 should clarify the root cause. Perhaps it's logm from some persistent slow ops? Cheers, Dan On Fri, Feb 24, 2023 at 7:36 AM Andrej Filipcic <andrej.filipcic@xxxxxx> wrote: > > > Hi, > > on our large ceph cluster with 60 servers, 1600 OSDs, we have observed > that small system nvmes are wearing out rapidly. Our monitoring shows > mon writes on average about 10MB/s to store.db. For small system nvmes > of 250GB and DWPD of ~1, this turns out to be too much, 0.8TB/day or > 1.5PB in 5 years, too much even for 3DWPD of the same capacity. > > Apart from replacing the drives with larger ones, more durable, > preferably both, do you have any suggestions if these writes can be > reduced? Actually, the mon writes match 0.15Hz rate of .sst file > creation of 64MB.... > > Best regards, > Andrej > > -- > _____________________________________________________________ > prof. dr. Andrej Filipcic, E-mail: Andrej.Filipcic@xxxxxx > Department of Experimental High Energy Physics - F9 > Jozef Stefan Institute, Jamova 39, P.o.Box 3000 > SI-1001 Ljubljana, Slovenia > Tel.: +386-1-477-3674 Fax: +386-1-425-7074 > ------------------------------------------------------------- > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |