Hi,
can you share the following output:
ceph -s
ceph health detail
ceph versions
ceph osd df tree
ceph osd dump
I see this line in the logs:
check_osdmap_features require_osd_release unknown -> octopus
which makes me wonder if you really run a Nautilus cluster.
Are your OSDs saturated? Check iostat for them.
If the OSDs tend to "flap" you could temporarily try to set 'ceph osd
set nodown' and see if the cluster stabilizes. If it does you'll need
to figure out what leads to the flapping. Did you check syslog of your
server and/or dmesg output?
Zitat von Mosharaf Hossain <mosharaf.hossain@xxxxxxxxxxxxxx>:
Hello
I am getting the following error for SSD osd.10.
root@ceph6:~# tailf /var/log/ceph/ceph-osd.10.log
2023-11-12T20:47:14.785+0600 7f17e9213d80 0 <cls>
/build/ceph-15.2.17/src/cls/hello/cls_hello.cc:312: loading cls_hello
2023-11-12T20:47:14.785+0600 7f17e9213d80 0 _get_class not permitted to
load lua
2023-11-12T20:47:14.785+0600 7f17e9213d80 1 osd.10 239394 warning: got an
error loading one or more classes: (1) Operation not permitted
2023-11-12T20:47:14.785+0600 7f17e9213d80 0 osd.10 239394 crush map has
features 432629239337189376, adjusting msgr requires for clients
2023-11-12T20:47:14.785+0600 7f17e9213d80 0 osd.10 239394 crush map has
features 432629239337189376 was 8705, adjusting msgr requires for mons
2023-11-12T20:47:14.785+0600 7f17e9213d80 0 osd.10 239394 crush map has
features 3314933000854323200, adjusting msgr requires for osds
2023-11-12T20:47:14.785+0600 7f17e9213d80 1 osd.10 239394
check_osdmap_features require_osd_release unknown -> octopus
2023-11-12T20:47:22.942+0600 7f17e9213d80 0 osd.10 239394 load_pgs
2023-11-12T20:47:53.767+0600 7f17e9213d80 0
bluestore(/var/lib/ceph/osd/ceph-10) log_latency slow operation observed
for next, latency = 25923326091ns
2023-11-12T20:49:16.757+0600 7f17e9213d80 0
bluestore(/var/lib/ceph/osd/ceph-10) log_latency slow operation observed
for next, latency = 74835459564ns
2023-11-12T20:50:14.991+0600 7f17e9213d80 0
bluestore(/var/lib/ceph/osd/ceph-10) log_latency slow operation observed
for next, latency = 42822161884ns
Regards
Mosharaf Hossain
Manager, Product Development
IT Division
Bangladesh Export Import Company Ltd.
Level-8, SAM Tower, Plot #4, Road #22, Gulshan-1, Dhaka-1212,Bangladesh
Tel: +880 9609 000 999, +880 2 5881 5559, Ext: 14191, Fax: +880 2 9895757
Cell: +8801787680828, Email: mosharaf.hossain@xxxxxxxxxxxxxx, Web:
www.bol-online.com
<https://www.google.com/url?q=http://www.bol-online.com&sa=D&source=hangouts&ust=1557908951423000&usg=AFQjCNGMxIuHSHsD3qO6y5JddpEZ0S592A>
On Sun, Nov 12, 2023 at 8:43 PM Mosharaf Hossain <
mosharaf.hossain@xxxxxxxxxxxxxx> wrote:
Hello
In my CEPH Nautilus cluster, some OSDs have unexpectedly gone offline,
despite all SSD disks appearing to be active on the node. What might be the
cause of this issue?
#ceph osd tree [showing disk for ceph6 status]
-11 19.20854 host ceph6
10 ssd 1.74609 osd.10 down 0 1.00000
35 ssd 1.74609 osd.35 up 1.00000 1.00000
40 ssd 1.74609 osd.40 down 0 1.00000
45 ssd 1.74609 osd.45 down 0 1.00000
50 ssd 1.74609 osd.50 down 0 1.00000
54 ssd 1.74609 osd.54 up 1.00000 1.00000
58 ssd 1.74609 osd.58 up 1.00000 1.00000
59 ssd 1.74609 osd.59 up 1.00000 1.00000
67 ssd 1.74660 osd.67 up 1.00000 1.00000
68 ssd 1.74660 osd.68 down 0 1.00000
90 ssd 1.74660 osd.90 up 1.00000 1.00000
root@ceph6:~# fdisk -l | grep 1920383410176
Disk /dev/sdb: 1.8 TiB, 1920383410176 bytes, 3750748848 sectors
Disk /dev/sda: 1.8 TiB, 1920383410176 bytes, 3750748848 sectors
Disk /dev/sdd: 1.8 TiB, 1920383410176 bytes, 3750748848 sectors
Disk /dev/sdc: 1.8 TiB, 1920383410176 bytes, 3750748848 sectors
Disk /dev/sdf: 1.8 TiB, 1920383410176 bytes, 3750748848 sectors
Disk /dev/sdg: 1.8 TiB, 1920383410176 bytes, 3750748848 sectors
Disk /dev/sdh: 1.8 TiB, 1920383410176 bytes, 3750748848 sectors
Disk /dev/sdi: 1.8 TiB, 1920383410176 bytes, 3750748848 sectors
Disk /dev/sde: 1.8 TiB, 1920383410176 bytes, 3750748848 sectors
Disk /dev/sdj: 1.8 TiB, 1920383410176 bytes, 3750748848 sectors
Disk /dev/sdk: 1.8 TiB, 1920383410176 bytes, 3750748848 sectors
Disk /dev/sdl: 1.8 TiB, 1920383410176 bytes, 3750748848 sectors
Regards
Mosharaf Hossain
Manager, Product Development
IT Division
Bangladesh Export Import Company Ltd.
Level-8, SAM Tower, Plot #4, Road #22, Gulshan-1, Dhaka-1212,Bangladesh
Tel: +880 9609 000 999, +880 2 5881 5559, Ext: 14191, Fax: +880 2 9895757
Cell: +8801787680828, Email: mosharaf.hossain@xxxxxxxxxxxxxx, Web:
www.bol-online.com
<https://www.google.com/url?q=http://www.bol-online.com&sa=D&source=hangouts&ust=1557908951423000&usg=AFQjCNGMxIuHSHsD3qO6y5JddpEZ0S592A>
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
