Hi all, After reboot one node, one OSD in other node has 'slow requests' and 'currently waiting for peered' a long time util restart this OSD. Is this a bug? See the attachment for more osd log. 2020-12-11 15:39:12.837391 7f3906fa2700 0 log_channel(cluster) log [WRN] : 15 slow requests, 1 included below; oldest blocked for > 85.733762 secs 2020-12-11 15:39:12.837400 7f3906fa2700 0 log_channel(cluster) log [WRN] : slow request 30.753996 seconds old, received at 2020-12-11 15:38:42.083351: osd_op(client.105567860.0:3107598 10.d393182f (undecoded) ack+ondisk+write+known_if_redirected e101930) currently waiting for peered 2020-12-11 15:39:19.996370 7f3906fa2700 0 log_channel(cluster) log [WRN] : 16 slow requests, 1 included below; oldest blocked for > 92.892750 secs 2020-12-11 15:39:19.996379 7f3906fa2700 0 log_channel(cluster) log [WRN] : slow request 30.655350 seconds old, received at 2020-12-11 15:38:49.340985: osd_op(client.64107841.0:99173823 10.dc5382f (undecoded) ack+read+known_if_redirected e101935) currently waiting for peered 2020-12-11 15:39:20.996497 7f3906fa2700 0 log_channel(cluster) log [WRN] : 17 slow requests, 2 included below; oldest blocked for > 93.892874 secs 2020-12-11 15:39:20.996504 7f3906fa2700 0 log_channel(cluster) log [WRN] : slow request 30.812449 seconds old, received at 2020-12-11 15:38:50.184011: osd_op(client.107537847.0:661426 10.71d5b82f (undecoded) ack+ondisk+write+known_if_redirected e101936) currently waiting for peered 2020-12-11 15:39:20.996507 7f3906fa2700 0 log_channel(cluster) log [WRN] : slow request 60.055144 seconds old, received at 2020-12-11 15:38:20.941316: osd_op(client.86166236.0:644979868 10.ec1182f (undecoded) ack+read+known_if_redirected e101919) currently waiting for peered 2020-12-11 15:39:21.996663 7f3906fa2700 0 log_channel(cluster) log [WRN] : 19 slow requests, 2 included below; oldest blocked for > 94.893016 secs 2020-12-11 15:39:21.996675 7f3906fa2700 0 log_channel(cluster) log [WRN] : slow request 30.947898 seconds old, received at 2020-12-11 15:38:51.048703: osd_op(client.106338809.0:4203455 10.6b2e582f (undecoded) ack+ondisk+write+known_if_redirected e101937) currently waiting for peered 2020-12-11 15:39:21.996679 7f3906fa2700 0 log_channel(cluster) log [WRN] : slow request 30.947137 seconds old, received at 2020-12-11 15:38:51.049464: osd_op(client.106338809.0:4203456 10.6b2e582f (undecoded) ack+ondisk+write+known_if_redirected e101937) currently waiting for peered 2020-12-11 15:39:23.996900 7f3906fa2700 0 log_channel(cluster) log [WRN] : 20 slow requests, 1 included below; oldest blocked for > 96.893269 secs 2020-12-11 15:39:23.996911 7f3906fa2700 0 log_channel(cluster) log [WRN] : slow request 30.175843 seconds old, received at 2020-12-11 15:38:53.821011: osd_op(client.85915739.0:221584769 10.e221f82f (undecoded) ack+read+known_if_redirected e101938) currently waiting for peered 2020-12-11 15:39:25.997168 7f3906fa2700 0 log_channel(cluster) log [WRN] : 21 slow requests, 1 included below; oldest blocked for > 98.893549 secs 2020-12-11 15:39:25.997175 7f3906fa2700 0 log_channel(cluster) log [WRN] : slow request 30.394852 seconds old, received at 2020-12-11 15:38:55.602282: osd_op(client.85915916.0:326668357 10.97bb82f (undecoded) ack+read+known_if_redirected e101939) currently waiting for peered 2020-12-11 15:39:26.997329 7f3906fa2700 0 log_channel(cluster) log [WRN] : 21 slow requests, 1 included below; oldest blocked for > 99.893677 secs 2020-12-11 15:39:26.997341 7f3906fa2700 0 log_channel(cluster) log [WRN] : slow request 60.617348 seconds old, received at 2020-12-11 15:38:26.379914: osd_op(client.85915739.0:221580487 10.e221f82f (undecoded) ack+read+known_if_redirected e101921) currently waiting for peered 2020-12-11 15:39:28.088682 7f3906fa2700 0 log_channel(cluster) log [WRN] : 21 slow requests, 1 included below; oldest blocked for > 100.985064 secs 2020-12-11 15:39:28.088688 7f3906fa2700 0 log_channel(cluster) log [WRN] : slow request 60.916264 seconds old, received at 2020-12-11 15:38:27.172385: osd_op(client.63799264.0:171265768 10.2dabb82f (undecoded) ack+read+known_if_redirected e101922) currently waiting for peered 2020-12-11 15:39:29.088805 7f3906fa2700 0 log_channel(cluster) log [WRN] : 22 slow requests, 1 included below; oldest blocked for > 101.985181 secs 2020-12-11 15:39:29.088813 7f3906fa2700 0 log_channel(cluster) log [WRN] : slow request 30.872263 seconds old, received at 2020-12-11 15:38:58.216504: osd_op(client.99944805.0:110150263 10.8474382f (undecoded) ack+ondisk+write+known_if_redirected e101939) currently waiting for peered 2020-12-11 15:39:30.088957 7f3906fa2700 0 log_channel(cluster) log [WRN] : 23 slow requests, 2 included below; oldest blocked for > 102.985309 secs 2020-12-11 15:39:30.088965 7f3906fa2700 0 log_channel(cluster) log [WRN] : slow request 30.242255 seconds old, received at 2020-12-11 15:38:59.846639: osd_op(client.92349012.0:326975621 10.afc8f82f (undecoded) ack+read+known_if_redirected e101939) currently waiting for peered Beset Regards， LiuGangbiao _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |