Be very careful with running "ceph pg repair". Have a look at this thread: http://thread.gmane.org/gmane.comp.file-systems.ceph.user/15185 -- Tomasz Kuzemko tomasz.kuzemko@xxxxxxx On Thu, Dec 11, 2014 at 10:57:22AM +0000, Luis Periquito wrote: > Hi, > > I've stopped OSD.16, removed the PG from the local filesystem and started > the OSD again. After ceph rebuilt the PG in the removed OSD I ran a > deep-scrub and the PG is still inconsistent. > > I'm running out of ideas on trying to solve this. Does this mean that all > copies of the object should also be inconsistent? Should I just try to > figure which object/bucket this belongs to and delete it/copy it again to > the ceph cluster? > > Also, do you know what the error message means? is it just some sort of > metadata for this object that isn't correct, not the object itself? > > On Wed, Dec 10, 2014 at 11:11 AM, Luis Periquito <periquito@xxxxxxxxx> > wrote: > > > Hi, > > > > In the last few days this PG (pool is .rgw.buckets) has been in error > > after running the scrub process. > > > > After getting the error, and trying to see what may be the issue (and > > finding none), I've just issued a ceph repair followed by a ceph > > deep-scrub. However it doesn't seem to have fixed the issue and it still > > remains. > > > > The relevant log from the OSD is as follows. > > > > 2014-12-10 09:38:09.348110 7f8f618be700 0 log [ERR] : 9.180 deep-scrub 0 > > missing, 1 inconsistent objects > > 2014-12-10 09:38:09.348116 7f8f618be700 0 log [ERR] : 9.180 deep-scrub 1 > > errors > > 2014-12-10 10:13:15.922065 7f8f618be700 0 log [INF] : 9.180 repair ok, 0 > > fixed > > 2014-12-10 10:55:27.556358 7f8f618be700 0 log [ERR] : 9.180 shard 6: soid > > 370cbf80/29145.4_xxx/head//9 missing attr _, missing attr _user.rgw.acl, > > missing attr _user.rgw.content_type, missing attr _user.rgw.etag, missing > > attr _user.rgw.idtag, missing attr _user.rgw.manifest, missing attr > > _user.rgw.x-amz-meta-md5sum, missing attr _user.rgw.x-amz-meta-stat, > > missing attr snapset > > 2014-12-10 10:56:50.597952 7f8f618be700 0 log [ERR] : 9.180 deep-scrub 0 > > missing, 1 inconsistent objects > > 2014-12-10 10:56:50.597957 7f8f618be700 0 log [ERR] : 9.180 deep-scrub 1 > > errors > > > > I'm running version firefly 0.80.7. > > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Attachment:
signature.asc
Description: Digital signature
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
