Hello Cephers, I am having some issues with two osds, which are either flapping or just crashing without recovering back. I've got a log file 100MB or so for these osds which has been generated in a couple of hours if anyone is interested. I am running firefly with the latest updates on Ubuntu 12.04 with the latest LTS kernel. Looking at the osd logs I see a bunch of these entries: 2014-09-26 15:24:08.998918 7f73cb194700 0 log [ERR] : 5.108 log bound mismatch, info (53757'2809698,54690'2817536] actual [53757'2809532,54690'2817536] followed by slow requests like these: 2014-09-26 15:24:16.798355 7f73e247c700 0 log [WRN] : slow request 31.463701 seconds old, received at 2014-09-26 15:23:45.334567: osd_op(client.37190249.0:6372257 rbd_data.3a0cd42ae8944a.000000000000280d [set-alloc-hint object_size 4194304 write_size 4194304,write 2203648~4096] 5.27e2bd53 ack+ondisk+write e54691) v4 currently waiting for subops from 8 2014-09-26 15:24:16.798358 7f73e247c700 0 log [WRN] : slow request 31.004246 seconds old, received at 2014-09-26 15:23:45.794022: osd_op(client.38862536.0:2001456 rbd_data.250f7505e5edd7.0000000000000f4f [stat,set-alloc-hint object_size 4194304 write_size 4194304,write 3813376~4096] 5.5a3d6aa3 ack+ondisk+write e54691) v4 currently waiting for missing object The cluster seems to suffer and the guest vms are running a bit with a lag. Any idea how to fix these issues? Cheers -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20140926/57303d43/attachment.htm>
 |