-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Exactly the problem. Thanks for the point in the right direction. - ---------------- Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Fri, Jan 15, 2016 at 10:28 AM, Sage Weil wrote: > On Thu, 14 Jan 2016, Robert LeBlanc wrote: >> We have a single VM that is acting odd. We had 7 SSD OSDs (out of 40) go >> down over a period of about 12 hours. These are a cache tier and have size >> 4, min_size 2. I'm not able to make heads or tails of the error and hoped >> someone here could help. >> >> 2016-01-14 23:09:54.559121 osd.136 [ERR] 13.503 copy from >> f8bedd03/rbd_data.48a6325f5e3f87.000000000000683d/head//13 to >> f8bedd03/rbd_data.48a6325f5e3f87.000000000000683d/head//13 data digest >> 0x92bc163c != source 0x8fe2d0a9 >> >> The PG fully recovered then the error was >> >> 2016-01-15 00:39:25.321469 osd.12 [ERR] 13.503 copy from >> f8bedd03/rbd_data.48a6325f5e3f87.000000000000683d/head//13 to >> f8bedd03/rbd_data.48a6325f5e3f87.000000000000683d/head//13 data digest >> 0x92bc163c != source 0x8fe2d0a9 >> >> A deep scrub of the PG comes back clean and a hash of the files on all OSDs >> match. The file system on this vm keeps going read only. >> >> The osd file system is EXT4 and this is 0.94.5. > > You're using cache tiering I take it? I think the error is in the base > tier, while the PG mentioned is the cache tier. > > sage -----BEGIN PGP SIGNATURE----- Version: Mailvelope v1.3.3 Comment: https://www.mailvelope.com wsFcBAEBCAAQBQJWmUlxCRDmVDuy+mK58QAAUVIP/0B7/0eMpV1lcEhXXLCE aNNBNSMwXevD7uRRVoqoUnjgqZExttMPNZhFij0MF5ztvJ88lycfb+1hClHk Z8agN+APhCVKFpbKm0bGf47EPs+BLYlTl8r7fmA2uIWcG0XO4+eZtjngsJON zZiM2ZE9URJ6GSvquCAADS1XDumAkve0CaJV0c6MOgj4nP1ktDiX0CzjdMF0 fpPc1hUZGAnCEevGGGEQmVcqXDZnUK3OsY2WCZiMCAXBiTQmNMVvJehM1rEg Ss/kC+VIyqzevPN+/r0STUve7UrkVXII19la5fBItSQt8btSjpEkzFFU1WmI 8ehX5KJwyKGH4NIrPCqjg+TGuUlTkkDO9LcEoj19x9I+Tzuyu/bSayo8FBvT 9SYkW7TzfF0JL+ed4fYCv7kK2OJBPR6uQZ8ABcLRXSbmtZa9oVKgHgTwvbgD jT+smWg9JB7bGrx7PkRYDQVpgr62Nbau/CFoUP1FWI8bVhStM3LQQQot8x7X pN9vFwAOwn5yvWLYi+7qmYYLyoOzkg8Ib7d1E3QfOzS86O3mCp14u++x1QBc 5YthsSxfvfs+fNmMKnCrz5YGGcCSntQgJRvOlQl8Oyp29xHVYTP1nqZSXhVi jC94GBmnX+3Z26JmnybVfZnELDbk9wsp2R0tN1Ai4JxpY0PUq/672PuAZDaO McAz =G1Lk -----END PGP SIGNATURE----- _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |