On Thu, Jan 7, 2016 at 1:17 AM, Evgeniy Firsov <Evgeniy.Firsov@xxxxxxxxxxx> wrote: > Hi, Devs, > > We hit an assert while doing scrub test. > Can you, please, verify if the test case is valid? > > The test: > 1. Start Jewel cluster. 2 nodes, 8 osds each. > 2. Start fio, pure write workload. > 3. Delete data of random pg: rm -rf /var/lib/ceph/osd/ceph-7/2.185_head/* > 4. Stop workload > 5. Do a scrub on that pg. Cluster active and clean after that. > 6. Restart the node with that pg. > 7. The OSD with that pg fails to recover and asserts with: > osd/PGLog.cc: 382: FAILED assert(objiter->second->version > > last_divergent_update) The OSD is passing scrub tests because of the file descriptor cache (so it still has its own fds open for the deleted files that it references), but then on restart everything gets wiped away and it discovers its data store is inconsistent. I believe your team has run into this before when designing data loss tests. :) -Greg -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
