Hi everyone, I have problem with swapping drive and partition names on reboot. My Ceph is Hammer on CentOS7, Dell R730 6xSSD (2xSSD OS RAID1 PERC, 4xSSD=Journal drives), 18x1.8T SAS for OSDs. Whenever I reboot, drives randomly seem to change names. This is extremely dangerous and frustrating when I've initially setup CEPH with ceph-deploy, zap, prepare and activate. It has happened that I've accidentally erased wrong disk too when e.g. /dev/sdX had become /dev/sdY. Please see an output below of how this drive swapping below appears SDC is shifted, indexes and drive names got shuffled. Ceph OSDs didn't come up properly. Please advice on how to get this corrected, with no more drive name shuffling. Can this be due to the PERC HW raid? thx will POST REBOOT 2 (expected outcome.. with sda,sdb,sdc,sdd as journal. sdw is a perc raid1) [cephnode3][INFO ] Running command: sudo /usr/sbin/ceph-disk list [cephnode3][DEBUG ] /dev/sda : [cephnode3][DEBUG ] /dev/sda1 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sda2 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sda3 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sda4 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sda5 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sdb : [cephnode3][DEBUG ] /dev/sdb1 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sdb2 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sdb3 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sdb4 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sdb5 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sdc : [cephnode3][DEBUG ] /dev/sdc1 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sdc2 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sdc3 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sdc4 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sdc5 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sdd : [cephnode3][DEBUG ] /dev/sdd1 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sdd2 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sdd3 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sdd4 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sdd5 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sde : [cephnode3][DEBUG ] /dev/sde1 ceph data, active, cluster ceph, osd.0 [cephnode3][DEBUG ] /dev/sdf : [cephnode3][DEBUG ] /dev/sdf1 ceph data, active, cluster ceph, osd.1 [cephnode3][DEBUG ] /dev/sdg : [cephnode3][DEBUG ] /dev/sdg1 ceph data, active, cluster ceph, osd.2 [cephnode3][DEBUG ] /dev/sdh : [cephnode3][DEBUG ] /dev/sdh1 ceph data, active, cluster ceph, osd.3 [cephnode3][DEBUG ] /dev/sdi : [cephnode3][DEBUG ] /dev/sdi1 ceph data, active, cluster ceph, osd.4 [cephnode3][DEBUG ] /dev/sdj : [cephnode3][DEBUG ] /dev/sdj1 ceph data, active, cluster ceph, osd.5 [cephnode3][DEBUG ] /dev/sdk : [cephnode3][DEBUG ] /dev/sdk1 ceph data, active, cluster ceph, osd.6 [cephnode3][DEBUG ] /dev/sdl : [cephnode3][DEBUG ] /dev/sdl1 ceph data, active, cluster ceph, osd.7 [cephnode3][DEBUG ] /dev/sdm : [cephnode3][DEBUG ] /dev/sdm1 other, xfs [cephnode3][DEBUG ] /dev/sdn : [cephnode3][DEBUG ] /dev/sdn1 ceph data, active, cluster ceph, osd.9 [cephnode3][DEBUG ] /dev/sdo : [cephnode3][DEBUG ] /dev/sdo1 ceph data, active, cluster ceph, osd.10 [cephnode3][DEBUG ] /dev/sdp : [cephnode3][DEBUG ] /dev/sdp1 ceph data, active, cluster ceph, osd.11 [cephnode3][DEBUG ] /dev/sdq : [cephnode3][DEBUG ] /dev/sdq1 ceph data, active, cluster ceph, osd.12 [cephnode3][DEBUG ] /dev/sdr : [cephnode3][DEBUG ] /dev/sdr1 ceph data, active, cluster ceph, osd.13 [cephnode3][DEBUG ] /dev/sds : [cephnode3][DEBUG ] /dev/sds1 ceph data, active, cluster ceph, osd.14 [cephnode3][DEBUG ] /dev/sdt : [cephnode3][DEBUG ] /dev/sdt1 ceph data, active, cluster ceph, osd.15 [cephnode3][DEBUG ] /dev/sdu : [cephnode3][DEBUG ] /dev/sdu1 ceph data, active, cluster ceph, osd.16 [cephnode3][DEBUG ] /dev/sdv : [cephnode3][DEBUG ] /dev/sdv1 ceph data, active, cluster ceph, osd.17 [cephnode3][DEBUG ] /dev/sdw : [cephnode3][DEBUG ] /dev/sdw1 other, xfs, mounted on / [cephnode3][DEBUG ] /dev/sdw2 swap, swap POST REBOOT 1: [cephnode3][DEBUG ] /dev/sda : [cephnode3][DEBUG ] /dev/sda1 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sda2 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sda3 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sda4 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sda5 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sdb : [cephnode3][DEBUG ] /dev/sdb1 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sdb2 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sdb3 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sdb4 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sdb5 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sdc : [cephnode3][DEBUG ] /dev/sdc1 ceph data, active, cluster ceph, osd.13 [cephnode3][DEBUG ] /dev/sdd : [cephnode3][DEBUG ] /dev/sdd1 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sdd2 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sdd3 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sdd4 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sdd5 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sde : [cephnode3][DEBUG ] /dev/sde1 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sde2 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sde3 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sde4 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sde5 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 [cephnode3][DEBUG ] /dev/sdf : [cephnode3][DEBUG ] /dev/sdf1 ceph data, active, cluster ceph, osd.0 [cephnode3][DEBUG ] /dev/sdg : [cephnode3][DEBUG ] /dev/sdg1 ceph data, active, cluster ceph, osd.14 [cephnode3][DEBUG ] /dev/sdh : [cephnode3][DEBUG ] /dev/sdh1 ceph data, active, cluster ceph, osd.1 [cephnode3][DEBUG ] /dev/sdi : [cephnode3][DEBUG ] /dev/sdi1 ceph data, active, cluster ceph, osd.2 [cephnode3][DEBUG ] /dev/sdj : [cephnode3][DEBUG ] /dev/sdj1 ceph data, active, cluster ceph, osd.3 [cephnode3][DEBUG ] /dev/sdk : [cephnode3][DEBUG ] /dev/sdk1 ceph data, active, cluster ceph, osd.4 [cephnode3][DEBUG ] /dev/sdl : [cephnode3][DEBUG ] /dev/sdl1 ceph data, active, cluster ceph, osd.15 [cephnode3][DEBUG ] /dev/sdm : [cephnode3][DEBUG ] /dev/sdm1 ceph data, active, cluster ceph, osd.5 [cephnode3][DEBUG ] /dev/sdn : [cephnode3][DEBUG ] /dev/sdn1 ceph data, active, cluster ceph, osd.6 [cephnode3][DEBUG ] /dev/sdo : [cephnode3][DEBUG ] /dev/sdo1 ceph data, active, cluster ceph, osd.7 [cephnode3][DEBUG ] /dev/sdp : [cephnode3][DEBUG ] /dev/sdp1 other, xfs [cephnode3][DEBUG ] /dev/sdq : [cephnode3][DEBUG ] /dev/sdq1 ceph data, active, cluster ceph, osd.9 [cephnode3][DEBUG ] /dev/sdr : [cephnode3][DEBUG ] /dev/sdr1 ceph data, active, cluster ceph, osd.16 [cephnode3][DEBUG ] /dev/sds : [cephnode3][DEBUG ] /dev/sds1 ceph data, active, cluster ceph, osd.10 [cephnode3][DEBUG ] /dev/sdt : [cephnode3][DEBUG ] /dev/sdt1 ceph data, active, cluster ceph, osd.11 [cephnode3][DEBUG ] /dev/sdu : [cephnode3][DEBUG ] /dev/sdu1 ceph data, active, cluster ceph, osd.12 [cephnode3][DEBUG ] /dev/sdv : [cephnode3][DEBUG ] /dev/sdv1 ceph data, active, cluster ceph, osd.17 [cephnode3][DEBUG ] /dev/sdw : [cephnode3][DEBUG ] /dev/sdw1 other, xfs, mounted on / [cephnode3][DEBUG ] /dev/sdw2 swap, swap _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |