Hi all!
I solved this situation with restarting the active mds. So the next mds took over and the error was gone.
This is somehow a strange situation. Similar to the situation when restarting primary osds when having scrub errors of pgs.
Maybe this should be researched a bit deeper.
Thanks all for this great storage solution!
Cheers,
Lars
Tue, 20 Aug 2019 07:30:11 +0200
Lars Täuber <taeuber@xxxxxxx> ==> ceph-users@xxxxxxxxxxxxxx :
> Hi there!
>
> Does anyone else have an idea what I could do to get rid of this error?
>
> BTW: it is the third time that the pg 20.0 is gone inconsistent.
> This is a pg from the metadata pool (cephfs).
> May this be related anyhow?
>
> # ceph health detail
> HEALTH_ERR 1 MDSs report damaged metadata; 1 scrub errors; Possible data damage: 1 pg inconsistent
> MDS_DAMAGE 1 MDSs report damaged metadata
> mdsmds3(mds.0): Metadata damage detected
> OSD_SCRUB_ERRORS 1 scrub errors
> PG_DAMAGED Possible data damage: 1 pg inconsistent
> pg 20.0 is active+clean+inconsistent, acting [9,27,15]
>
>
> Best regards,
> Lars
>
>
> Mon, 19 Aug 2019 13:51:59 +0200
> Lars Täuber <taeuber@xxxxxxx> ==> Paul Emmerich <paul.emmerich@xxxxxxxx> :
> > Hi Paul,
> >
> > thanks for the hint.
> >
> > I did a recursive scrub from "/". The log says there where some inodes with bad backtraces repaired. But the error remains.
> > May this have something to do with a deleted file? Or a file within a snapshot?
> >
> > The path told by
> >
> > # ceph tell mds.mds3 damage ls
> > 2019-08-19 13:43:04.608 7f563f7f6700 0 client.894552 ms_handle_reset on v2:192.168.16.23:6800/176704036
> > 2019-08-19 13:43:04.624 7f56407f8700 0 client.894558 ms_handle_reset on v2:192.168.16.23:6800/176704036
> > [
> > {
> > "damage_type": "backtrace",
> > "id": 3760765989,
> > "ino": 1099518115802,
> > "path": "~mds0/stray7/100005161f7/dovecot.index.backup"
> > }
> > ]
> >
> > starts a bit strange to me.
> >
> > Are the snapshots also repaired with a recursive repair operation?
> >
> > Thanks
> > Lars
> >
> >
> > Mon, 19 Aug 2019 13:30:53 +0200
> > Paul Emmerich <paul.emmerich@xxxxxxxx> ==> Lars Täuber <taeuber@xxxxxxx> :
> > > Hi,
> > >
> > > that error just says that the path is wrong. I unfortunately don't
> > > know the correct way to instruct it to scrub a stray path off the top
> > > of my head; you can always run a recursive scrub on / to go over
> > > everything, though
> > >
> > >
> > > Paul
> > >
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
--
Informationstechnologie
Berlin-Brandenburgische Akademie der Wissenschaften
Jägerstraße 22-23 10117 Berlin
Tel.: +49 30 20370-352 http://www.bbaw.de
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
