Oh that is very good to hear. So how should I be cleaning this up? I read some post of Sage that scrubbing is not taking care of this. Should I be dumping the logs with objects like 17:e80576a8:::rbd_data.2cc7df2ae8944a.00000000000009f8:27 and try to delete these manually? -----Original Message----- From: Paul Emmerich [mailto:paul.emmerich@xxxxxxxx] Sent: donderdag 12 april 2018 11:04 To: Marc Roos Cc: ceph-users Subject: Re: Ceph scrub logs: _scan_snaps no head for $object? Usually the problem is not that you are missing snapshot data, but that you got too many snapshots, so your snapshots are probably fine. You're just wasting space. Paul 2018-04-10 16:07 GMT+02:00 Marc Roos <M.Roos@xxxxxxxxxxxxxxxxx>: Hi Paul, This is a small test cluster, and the rbd pool is replicated. I am hardly using any clients on the cluster. Furthermore I have been the only one creating the snapshots and I know for sure that I was not trying to delete them. If so I have been doing this on one of the ceph nodes. I have these issues on images with create_timestamp: Tue Jul 18 20:51:40 2017 create_timestamp: Fri Sep 1 13:55:25 2017 create_timestamp: Fri Sep 1 13:59:10 2017 create_timestamp: Wed Jan 3 16:38:57 2018 Updates have been done in February, so theoretically I should not be seeing these than any more? Feb 21 15:13:35 Updated: 2:ceph-osd-12.2.3-0.el7.x86_64 Feb 28 13:33:27 Updated: 2:ceph-osd-12.2.4-0.el7.x86_64 How can I determine what snapshot is bad of this image? Should this snapshot be considered lost? And is deleting this snapshot the only way to fix this? -----Original Message----- From: Paul Emmerich [mailto:paul.emmerich@xxxxxxxx] Sent: dinsdag 10 april 2018 20:14 To: Marc Roos Cc: ceph-users Subject: Re: Ceph scrub logs: _scan_snaps no head for $object? Hi, you'll usually see this if there are "orphaned" snapshot objects. One common cause for this are pre-12.2.2 clients trying to delete RBD snapshots with a data pool (i.e., erasure coded pools) They send the snapshot requests to the wrong pool and you end up with lots of problems. Paul 2018-04-09 16:55 GMT+02:00 Marc Roos <M.Roos@xxxxxxxxxxxxxxxxx>: I have this on a rbd pool with images/snapshots that have been created in Luminous > Hi Stefan, Mehmet, > > Are these clusters that were upgraded from prior versions, or fresh > luminous installs? > > > This message indicates that there is a stray clone object with no > associated head or snapdir object. That normally should never > happen--it's presumably the result of a (hopefully old) bug. The scrub > process doesn't even clean them up, which maybe says something about how > common it is/was... > > sage > _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com <http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com> <http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com <http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com> > -- -- Paul Emmerich croit GmbH Freseniusstr. 31h 81247 München www.croit.io Tel: +49 89 1896585 90 -- -- Paul Emmerich croit GmbH Freseniusstr. 31h 81247 München www.croit.io Tel: +49 89 1896585 90 _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com