Re: OSD Crash with scan_min and scan_max values reduced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



So basically the issue - http://tracker.ceph.com/issues/4698

osd suicide timeout

On Mon, Feb 22, 2016 at 7:06 PM, M Ranga Swami Reddy
<swamireddy@xxxxxxxxx> wrote:
> Hello,
> I have reduced the scan_min and scan_max as below. After the below
> change, during the scrubbing, got the op_tp_thread time out after 15.
> After some time, OSDs crashed also... Any suggestions will be
> helpful... Thanking you.
> ==
> -osd_backfill_scan_min = 64
> -osd_backfill_scan_max = 512
> +osd_backfill_scan_min = 16
> +osd_backfill_scan_max = 32
> osd_op_thread_timeout = 15
> osd_recovery_thread_timeout = 30
> ==
>
> emerg:  2016-02-22 02:48:24.301522 7f8ee3096700  1 heartbeat_map
> is_healthy 'OSD::op_tp thread 0x7f8edf08e700' had timed out after 15
>
> 2016-02-22T02:54:20.325657+00:00 emerg:  --- begin dump of recent events ---
>
>
>
> 2016-02-22T02:54:20.435546+00:00 emerg:  2016-02-22 02:54:20.315656
> 7f31a2166700  1 heartbeat_map reset_timeout 'OSD::op_tp thread
> 0x7f31a2166700' had suicide timed out after 150
>
>
>
> 2016-02-22T02:54:20.456737+00:00 emerg:  2016-02-22 02:54:20.237829
> 7fa09e05f700 -1 common/HeartbeatMap.cc: In function 'bool
> ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d*, const char*,
> time_t)' thread 7fa09e05f700 time 2016-02-22 02:54:20.187089
>
> common/HeartbeatMap.cc: 79: FAILED assert(0 == "hit suicide timeout")
>
>
>
> ceph version 0.80.7 (6c0127fcb58008793d3c8b62d925bc91963672a3)
>
> 1: (ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d*, char const*,
> long)+0x107) [0x9c5757]
>
> 2: (ceph::HeartbeatMap::is_healthy()+0xa7) [0x9c6107]
>
> 3: (OSD::handle_osd_ping(MOSDPing*)+0x81b) [0x647a5b]
>
> 4: (OSD::heartbeat_dispatch(Message*)+0x51b) [0x648d0b]
>
> 5: (DispatchQueue::entry()+0x549) [0xb27269]
>
> 6: (DispatchQueue::DispatchThread::entry()+0xd) [0xa618ad]
>
> 7: (()+0x7e9a) [0x7fa0b4e77e9a]
>
> 8: (clone()+0x6d) [0x7fa0b3d708bd]
>
> NOTE: a copy of the executable, or `objdump -rdS <executable>` is
> needed to interpret this.
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux