Hi, This problem was reported some time ago [1] though it seemed then the root cause was not identified. Now we have a customer that is experiencing the same problem so we did some investigation. The issue is that in a bluestore/filestore mixed cluster the deep scrub may incorrectly report missing objects due to object sorting order is different on bluestore and filestore osds. Looking at the cases we have it happens for objects that have the same hash. When building the sorting key the bluestore escapes the object name string, while the filestore sorts with the raw object name, which may result in different order. I pushed a PR for review that changes the sorting order for the filestore [2]. Though using this solution would fix a cluster of mixed bluestore and upgraded filestore osds but introduce the same issue for a cluster of mixed old version and new version filestore osds. [1] https://tracker.ceph.com/issues/43174 [2] https://github.com/ceph/ceph/pull/35938 -- Mykola Golub _______________________________________________ Dev mailing list -- dev@xxxxxxx To unsubscribe send an email to dev-leave@xxxxxxx
