Ceph crush map randomly changes for one host

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



My Tree is supposed to look like below but it keeps changing to the map further below. Notice the drives moving from chassis osd3-shelf1 chassis to host ceph-osd3. Does anyone know why this may happen?

 

I wrote a script to monitor for this and to place the osds back where they belong if they notice the change, but this should obviously not be necessary. I would appreciate any help with this.

 

 

 

ID  CLASS WEIGHT   TYPE NAME                      STATUS REWEIGHT PRI-AFF

-62       14.55199 root osd3-shelf2                                       

-60       14.55199     chassis ceph-osd3-shelf2                          

  3   hdd  1.81898         osd.3                    down  1.00000 1.00000

 40   hdd  1.81898         osd.40                   down  1.00000 1.00000

 41   hdd  1.81898         osd.41                   down  1.00000 1.00000

 42   hdd  1.81898         osd.42                   down  1.00000 1.00000

 43   hdd  1.81898         osd.43                   down  1.00000 1.00000

 44   hdd  1.81898         osd.44                   down  1.00000 1.00000

 45   hdd  1.81898         osd.45                   down  1.00000 1.00000

 46   hdd  1.81898         osd.46                   down  1.00000 1.00000

-58        1.71599 root osd3-internal                                     

-54        1.71599     chassis ceph-osd3-internal                        

 34   hdd  0.42899         osd.34                   down  1.00000 1.00000

 35   hdd  0.42899         osd.35                   down  1.00000 1.00000

 36   hdd  0.42899         osd.36                   down  1.00000 1.00000

 37   hdd  0.42899         osd.37                   down  1.00000 1.00000

-50       14.55199 root osd3-shelf1                                      

-56       14.55199     chassis ceph-osd3-shelf1                           

 21   hdd  1.81898         osd.21                   down  1.00000 1.00000

 22   hdd  1.81898         osd.22                   down  1.00000 1.00000

 23   hdd  1.81898         osd.23                   down  1.00000 1.00000

 24   hdd  1.81898         osd.24                   down  1.00000 1.00000

 25   hdd  1.81898         osd.25                   down  1.00000 1.00000

 28   hdd  1.81898         osd.28                   down  1.00000 1.00000

 29   hdd  1.81898         osd.29                   down  1.00000 1.00000

 31   hdd  1.81898         osd.31                   down  1.00000 1.00000

 -7        5.45695 host ceph-osd3                                        

 26   hdd  1.81898     osd.26                       down        0 1.00000

 27   hdd  1.81898     osd.27                       down        0 1.00000

 30   hdd  1.81898     osd.30                       down        0 1.00000

 -1       47.21199 root default                                           

-40       23.59000     rack mainehall                                    

 -3       23.59000         host ceph-osd1                                

  0   hdd  1.81898             osd.0                  up  1.00000 1.00000

  1   hdd  1.81898             osd.1                  up  1.00000 1.00000

  2   hdd  1.81898             osd.2                  up  1.00000 1.00000

  4   hdd  1.81898             osd.4                  up  1.00000 1.00000

  5   hdd  1.81898             osd.5                  up  0.90002 1.00000

  6   hdd  1.81898             osd.6                  up  1.00000 1.00000

  7   hdd  1.81898             osd.7                  up  1.00000 1.00000

  8   hdd  1.81898             osd.8                  up  1.00000 1.00000

  9   hdd  1.81898             osd.9                  up  1.00000 1.00000

 10   hdd  1.81898             osd.10                 up  0.95001 1.00000

 33   hdd  1.76099             osd.33                 up  1.00000 1.00000

 38   hdd  3.63899             osd.38                 up  1.00000 1.00000

-42       23.62199     rack rangleyhall                                  

 -5       23.62199         host ceph-osd2                                

 11   hdd  1.81898             osd.11                 up  1.00000 1.00000

 12   hdd  1.81898             osd.12                 up  0.90002 1.00000

 13   hdd  1.81898             osd.13                 up  1.00000 1.00000

 14   hdd  1.81898             osd.14                 up  1.00000 1.00000

 15   hdd  1.81898             osd.15                 up  1.00000 1.00000

 16   hdd  1.81898             osd.16                 up  1.00000 1.00000

 17   hdd  1.81898             osd.17                 up  1.00000 1.00000

 18   hdd  1.81898             osd.18                 up  0.90002 1.00000

 19   hdd  1.81898             osd.19                 up  1.00000 1.00000

 20   hdd  1.81898             osd.20                 up  1.00000 1.00000

 32   hdd  1.79300             osd.32                 up  0.95001 1.00000

 39   hdd  3.63899             osd.39                 up  1.00000 1.00000

 

 

But it keeps changing to this:

ID  CLASS WEIGHT   TYPE NAME                      STATUS REWEIGHT PRI-AFF

-62       14.55199 root osd3-shelf2                                      

-60       14.55199     chassis ceph-osd3-shelf2                          

  3   hdd  1.81898         osd.3                    down  1.00000 1.00000

 40   hdd  1.81898         osd.40                   down  1.00000 1.00000

 41   hdd  1.81898         osd.41                   down  1.00000 1.00000

 42   hdd  1.81898         osd.42                   down  1.00000 1.00000

 43   hdd  1.81898         osd.43                   down  1.00000 1.00000

 44   hdd  1.81898         osd.44                   down  1.00000 1.00000

 45   hdd  1.81898         osd.45                   down  1.00000 1.00000

 46   hdd  1.81898         osd.46                   down  1.00000 1.00000

-58        1.71599 root osd3-internal                                    

-54        1.71599     chassis ceph-osd3-internal                        

 34   hdd  0.42899         osd.34                   down  1.00000 1.00000

 35   hdd  0.42899         osd.35                   down  1.00000 1.00000

 36   hdd  0.42899         osd.36                   down  1.00000 1.00000

 37   hdd  0.42899         osd.37                   down  1.00000 1.00000

-50       14.55199 root osd3-shelf1                                      

-56       14.55199     chassis ceph-osd3-shelf1                          

 21   hdd  1.81898         osd.21                   down  1.00000 1.00000

 22   hdd  1.81898         osd.22                   down  1.00000 1.00000

 23   hdd  1.81898         osd.23                   down  1.00000 1.00000

 24   hdd  1.81898         osd.24                   down  1.00000 1.00000

 25   hdd  1.81898         osd.25                   down  1.00000 1.00000

-7        5.45695 host ceph-osd3                                        

 26   hdd  1.81898     osd.26                       down        0 1.00000

 27   hdd  1.81898     osd.27                       down        0 1.00000

30   hdd  1.81898     osd.30                       down        0 1.00000

28   hdd  1.81898         osd.28                   down  1.00000 1.00000

29   hdd  1.81898         osd.29                   down  1.00000 1.00000

31   hdd  1.81898         osd.31                   down  1.00000 1.00000

 

-1       47.21199 root default                                          

-40       23.59000     rack mainehall                                    

 -3       23.59000         host ceph-osd1                                

  0   hdd  1.81898             osd.0                  up  1.00000 1.00000

  1   hdd  1.81898             osd.1                  up  1.00000 1.00000

  2   hdd  1.81898             osd.2                  up  1.00000 1.00000

  4   hdd  1.81898             osd.4                  up  1.00000 1.00000

  5   hdd  1.81898             osd.5                  up  0.90002 1.00000

  6   hdd  1.81898             osd.6                  up  1.00000 1.00000

  7   hdd  1.81898             osd.7                  up  1.00000 1.00000

  8   hdd  1.81898             osd.8                  up  1.00000 1.00000

  9   hdd  1.81898             osd.9                  up  1.00000 1.00000

 10   hdd  1.81898             osd.10                 up  0.95001 1.00000

 33   hdd  1.76099             osd.33                 up  1.00000 1.00000

 38   hdd  3.63899             osd.38                 up  1.00000 1.00000

-42       23.62199     rack rangleyhall                                  

 -5       23.62199         host ceph-osd2                                

 11   hdd  1.81898             osd.11                 up  1.00000 1.00000

 12   hdd  1.81898             osd.12                 up  0.90002 1.00000

 13   hdd  1.81898             osd.13                 up  1.00000 1.00000

 14   hdd  1.81898             osd.14                 up  1.00000 1.00000

 15   hdd  1.81898             osd.15                 up  1.00000 1.00000

 16   hdd  1.81898             osd.16                 up  1.00000 1.00000

 17   hdd  1.81898             osd.17                 up  1.00000 1.00000

 18   hdd  1.81898             osd.18                 up  0.90002 1.00000

 19   hdd  1.81898             osd.19                 up  1.00000 1.00000

 20   hdd  1.81898             osd.20                 up  1.00000 1.00000

 32   hdd  1.79300             osd.32                 up  0.95001 1.00000

 39   hdd  3.63899             osd.39                 up  1.00000 1.00000

Robert Pelletier, IT and Security Specialist

Eastern Maine Community College
(207) 974-4782 | 354 Hogan Rd., Bangor, ME 04401

 

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux