Re: ERR scrub mismatch

[Marco Aroldi <marco.aroldi@xxxxxxxxx>]

Hello,

the errors are not resolved. Here is what I tried so far, without luck:

I have added a sixth monitor (ceph-mon06), then I deleted the first one (ceph-mon01)

The mon IDs shift back (mon02 was ID1, now is 0 and so on...)

This is the actual monmap:
0: 192.168.50.21:6789/0 mon.ceph-mon02
1: 192.168.50.22:6789/0 mon.ceph-mon03
2: 192.168.50.23:6789/0 mon.ceph-mon04
3: 192.168.50.24:6789/0 mon.ceph-mon05
4: 192.168.50.25:6789/0 mon.ceph-mon06

Since this is a multi-mds setup (1 active, 3 standby), I've issued the command
ceph fs set cephfs01 max_mds 2

Now we have 2 mds active and 2 standby

This is the mon log excerpt:

https://paste.ubuntu.com/p/2CHyDzxfSk/

I'm tempted to delete mon.0 (ceph-mon02 in my setup)

Please, any thoughts?

Thanks

Marco

Il giorno gio 8 nov 2018 alle ore 10:54 Marco Aroldi <marco.aroldi@xxxxxxxxx> ha scritto:

Hello,

Since upgrade from Jewel to Luminous 12.2.8, in the logs are reported some errors related to "scrub mismatch", every day at the same time.

I have 5 mon (from mon.0 to mon.4) and I need help to indentify and recover from this problem.

This is the log:
2018-11-07 15:13:53.808128 [ERR]  mon.4 ScrubResult(keys {logm=46,mds_health=29,mds_metadata=1,mdsmap=24} crc {logm=1239992787,mds_health=3182263811,mds_metadata=3704185590,mdsmap=1114086003})
2018-11-07 15:13:53.808095 [ERR]  mon.0 ScrubResult(keys {logm=46,mds_health=30,mds_metadata=1,mdsmap=23} crc {logm=1239992787,mds_health=1194056063,mds_metadata=3704185590,mdsmap=3259702002})
2018-11-07 15:13:53.808061 [ERR]  scrub mismatch
2018-11-07 15:13:53.808026 [ERR]  mon.3 ScrubResult(keys {logm=46,mds_health=31,mds_metadata=1,mdsmap=22} crc {logm=1239992787,mds_health=807938287,mds_metadata=3704185590,mdsmap=662277977})
2018-11-07 15:13:53.807970 [ERR]  mon.0 ScrubResult(keys {logm=46,mds_health=30,mds_metadata=1,mdsmap=23} crc {logm=1239992787,mds_health=1194056063,mds_metadata=3704185590,mdsmap=3259702002})
2018-11-07 15:13:53.807939 [ERR]  scrub mismatch
2018-11-07 15:13:53.807916 [ERR]  mon.2 ScrubResult(keys {logm=46,mds_health=31,mds_metadata=1,mdsmap=22} crc {logm=1239992787,mds_health=807938287,mds_metadata=3704185590,mdsmap=662277977})
2018-11-07 15:13:53.807882 [ERR]  mon.0 ScrubResult(keys {logm=46,mds_health=30,mds_metadata=1,mdsmap=23} crc {logm=1239992787,mds_health=1194056063,mds_metadata=3704185590,mdsmap=3259702002})
2018-11-07 15:13:53.807844 [ERR]  scrub mismatch

Any help will be appreciated

Thanks

Marco

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com