Re: Troubleshoot blocked OSDs

[Lincoln Bryant <lincolnb@xxxxxxxxxxxx>]

Hi Brian,

The first thing you can do is “ceph health detail”, which should give you some more information about which OSD(s) have blocked requests.

If it’s isolated to one OSD in particular, perhaps use iostat to check utilization and/or smartctl to check health. 

—Lincoln

On Apr 28, 2016, at 2:26 PM, Andrus, Brian Contractor <bdandrus@xxxxxxx> wrote:

All,

 

I have a small ceph cluster with 4 OSDs and 3 MONs on 4 systems.

I was rsyncing about 50TB of files and things get very slow. To the point I stopped the rsync, but even with everything stopped, I see:

 

health HEALTH_WARN

            80 requests are blocked > 32 sec

 

The number was as high as 218, but they seem to be draining down.

I see no issues on any of the systems, CPU load is low, memory usage is low.

 

How do I go about finding why a request is blocked for so long? These have been hitting >500 seconds for block time.

 

Brian Andrus

ITACS/Research Computing

Naval Postgraduate School

Monterey, California

voice: 831-656-6238

 

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com