Hello THis osd got up but getting following error and it getting up and down. 2023-11-12T20:53:04.652+0600 7f17e3e9d700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f17c2dff700' had timed out after 15 2023-11-12T20:53:04.652+0600 7f17e3e9d700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f17c4602700' had timed out after 15 2023-11-12T20:53:04.652+0600 7f17e3e9d700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f17c5604700' had timed out after 15 2023-11-12T20:53:04.652+0600 7f17e3e9d700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f17c5e05700' had timed out after 15 2023-11-12T20:53:04.652+0600 7f17e2e9b700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f17c05fa700' had timed out after 15 2023-11-12T20:53:04.652+0600 7f17e2e9b700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f17c15fc700' had timed out after 15 2023-11-12T20:53:04.652+0600 7f17e2e9b700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f17c1dfd700' had timed out after 15 2023-11-12T20:53:04.652+0600 7f17e2e9b700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f17c2dff700' had timed out after 15 2023-11-12T20:53:04.652+0600 7f17e2e9b700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f17c4602700' had timed out after 15 2023-11-12T20:53:04.652+0600 7f17e2e9b700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f17c5604700' had timed out after 15 2023-11-12T20:53:04.652+0600 7f17e2e9b700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f17c5e05700' had timed out after 15 Regards Mosharaf Hossain Manager, Product Development IT Division Bangladesh Export Import Company Ltd. Level-8, SAM Tower, Plot #4, Road #22, Gulshan-1, Dhaka-1212,Bangladesh Tel: +880 9609 000 999, +880 2 5881 5559, Ext: 14191, Fax: +880 2 9895757 Cell: +8801787680828, Email: mosharaf.hossain@xxxxxxxxxxxxxx, Web: www.bol-online.com <https://www.google.com/url?q=http://www.bol-online.com&sa=D&source=hangouts&ust=1557908951423000&usg=AFQjCNGMxIuHSHsD3qO6y5JddpEZ0S592A> On Sun, Nov 12, 2023 at 8:51 PM Mosharaf Hossain < mosharaf.hossain@xxxxxxxxxxxxxx> wrote: > Hello > I am getting the following error for SSD osd.10. > > root@ceph6:~# tailf /var/log/ceph/ceph-osd.10.log > 2023-11-12T20:47:14.785+0600 7f17e9213d80 0 <cls> > /build/ceph-15.2.17/src/cls/hello/cls_hello.cc:312: loading cls_hello > 2023-11-12T20:47:14.785+0600 7f17e9213d80 0 _get_class not permitted to > load lua > 2023-11-12T20:47:14.785+0600 7f17e9213d80 1 osd.10 239394 warning: got an > error loading one or more classes: (1) Operation not permitted > 2023-11-12T20:47:14.785+0600 7f17e9213d80 0 osd.10 239394 crush map has > features 432629239337189376, adjusting msgr requires for clients > 2023-11-12T20:47:14.785+0600 7f17e9213d80 0 osd.10 239394 crush map has > features 432629239337189376 was 8705, adjusting msgr requires for mons > 2023-11-12T20:47:14.785+0600 7f17e9213d80 0 osd.10 239394 crush map has > features 3314933000854323200, adjusting msgr requires for osds > 2023-11-12T20:47:14.785+0600 7f17e9213d80 1 osd.10 239394 > check_osdmap_features require_osd_release unknown -> octopus > 2023-11-12T20:47:22.942+0600 7f17e9213d80 0 osd.10 239394 load_pgs > 2023-11-12T20:47:53.767+0600 7f17e9213d80 0 > bluestore(/var/lib/ceph/osd/ceph-10) log_latency slow operation observed > for next, latency = 25923326091ns > 2023-11-12T20:49:16.757+0600 7f17e9213d80 0 > bluestore(/var/lib/ceph/osd/ceph-10) log_latency slow operation observed > for next, latency = 74835459564ns > 2023-11-12T20:50:14.991+0600 7f17e9213d80 0 > bluestore(/var/lib/ceph/osd/ceph-10) log_latency slow operation observed > for next, latency = 42822161884ns > > > > Regards > Mosharaf Hossain > Manager, Product Development > IT Division > > Bangladesh Export Import Company Ltd. > > Level-8, SAM Tower, Plot #4, Road #22, Gulshan-1, Dhaka-1212,Bangladesh > > Tel: +880 9609 000 999, +880 2 5881 5559, Ext: 14191, Fax: +880 2 9895757 > > Cell: +8801787680828, Email: mosharaf.hossain@xxxxxxxxxxxxxx, Web: > www.bol-online.com > > <https://www.google.com/url?q=http://www.bol-online.com&sa=D&source=hangouts&ust=1557908951423000&usg=AFQjCNGMxIuHSHsD3qO6y5JddpEZ0S592A> > > > > On Sun, Nov 12, 2023 at 8:43 PM Mosharaf Hossain < > mosharaf.hossain@xxxxxxxxxxxxxx> wrote: > >> Hello >> In my CEPH Nautilus cluster, some OSDs have unexpectedly gone offline, >> despite all SSD disks appearing to be active on the node. What might be the >> cause of this issue? >> >> #ceph osd tree [showing disk for ceph6 status] >> -11 19.20854 host ceph6 >> 10 ssd 1.74609 osd.10 down 0 1.00000 >> 35 ssd 1.74609 osd.35 up 1.00000 1.00000 >> 40 ssd 1.74609 osd.40 down 0 1.00000 >> 45 ssd 1.74609 osd.45 down 0 1.00000 >> 50 ssd 1.74609 osd.50 down 0 1.00000 >> 54 ssd 1.74609 osd.54 up 1.00000 1.00000 >> 58 ssd 1.74609 osd.58 up 1.00000 1.00000 >> 59 ssd 1.74609 osd.59 up 1.00000 1.00000 >> 67 ssd 1.74660 osd.67 up 1.00000 1.00000 >> 68 ssd 1.74660 osd.68 down 0 1.00000 >> 90 ssd 1.74660 osd.90 up 1.00000 1.00000 >> >> root@ceph6:~# fdisk -l | grep 1920383410176 >> Disk /dev/sdb: 1.8 TiB, 1920383410176 bytes, 3750748848 sectors >> Disk /dev/sda: 1.8 TiB, 1920383410176 bytes, 3750748848 sectors >> Disk /dev/sdd: 1.8 TiB, 1920383410176 bytes, 3750748848 sectors >> Disk /dev/sdc: 1.8 TiB, 1920383410176 bytes, 3750748848 sectors >> Disk /dev/sdf: 1.8 TiB, 1920383410176 bytes, 3750748848 sectors >> Disk /dev/sdg: 1.8 TiB, 1920383410176 bytes, 3750748848 sectors >> Disk /dev/sdh: 1.8 TiB, 1920383410176 bytes, 3750748848 sectors >> Disk /dev/sdi: 1.8 TiB, 1920383410176 bytes, 3750748848 sectors >> Disk /dev/sde: 1.8 TiB, 1920383410176 bytes, 3750748848 sectors >> Disk /dev/sdj: 1.8 TiB, 1920383410176 bytes, 3750748848 sectors >> Disk /dev/sdk: 1.8 TiB, 1920383410176 bytes, 3750748848 sectors >> Disk /dev/sdl: 1.8 TiB, 1920383410176 bytes, 3750748848 sectors >> >> >> >> Regards >> Mosharaf Hossain >> Manager, Product Development >> IT Division >> >> Bangladesh Export Import Company Ltd. >> >> Level-8, SAM Tower, Plot #4, Road #22, Gulshan-1, Dhaka-1212,Bangladesh >> >> Tel: +880 9609 000 999, +880 2 5881 5559, Ext: 14191, Fax: +880 2 9895757 >> >> Cell: +8801787680828, Email: mosharaf.hossain@xxxxxxxxxxxxxx, Web: >> www.bol-online.com >> >> <https://www.google.com/url?q=http://www.bol-online.com&sa=D&source=hangouts&ust=1557908951423000&usg=AFQjCNGMxIuHSHsD3qO6y5JddpEZ0S592A> >> >> _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |