check mtu between nodes first, ping with mtu size to check it. Vào Th 2, 17 thg 6, 2024 vào lúc 22:59 Sarunas Burdulis < sarunas@xxxxxxxxxxxxxxxxxx> đã viết: > Hi, > > 6 host 16 OSD cluster here, all SATA SSDs. All Ceph daemons version > 18.2.2. Host OS is Ubuntu 24.04. Intel X540 10Gb/s interfaces for > cluster network. All is fine while using a 1Gb/s switch. When moved to > 10Gb/s switch (Netgear XS712T), OSDs, one-by-one start failing heartbeat > checks and are marked as 'down' until only 3 or 4 OSDs remain up. By > then cluster is unusable (slow ops, PGs inactive). > > Here is a sample sequence from the log of one of the OSDs: > > ceph-osd[23402]: osd.3 77434 heartbeat_check: no reply from > 129.170.x.x:6802 osd.13 ever on either front or back > > ceph-osd[23402]: log_channel(cluster) log [WRN] : 101 slow requests (by > type [ 'delayed' : 101 ] most affected pool [ 'default.rgw.log' : 96 ]) > > ceph-osd[23402]: log_channel(cluster) log [WRN] : Monitor daemon marked > osd.3 down, but it is still running > > ceph-osd[23402]: log_channel(cluster) log [DBG] : map e77442 wrongly > marked me down at e77441 > > ceph-osd[23402]: osd.3 77442 start_waiting_for_healthy > > ceph-osd[23402]: osd.3 77434 is_healthy false -- only 0/10 up peers > (less than 33%) > > ceph-osd[23402]: osd.3 77434 not healthy; waiting to boot > > OSD service container keeps running, but it is not booted. > > Has anyone experienced this? Any ideas on what should be fixed? Please > let me know what other info would be useful. > > Best regards, > -- > Sarunas Burdulis > Dartmouth Mathematics > math.dartmouth.edu/~sarunas > > · https://useplaintext.email · > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > -- Trân trọng, ---------------------------------------------------------------------------- *Tran Thanh Phong* Email: tranphong079@xxxxxxxxx Skype: tranphong079 _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |