ceph balancer - Some osds belong to multiple subtrees

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,
tried to enable the ceph balancer on a 12.2.12 cluster and got this: 

mgr[balancer] Some osds belong to multiple subtrees: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290, 291, 292, 293, 294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304, 305]

I'm not aware of any additional subtree - maybe someone can enlighten me: 

ceph balancer status
{
    "active": true, 
    "plans": [], 
    "mode": "crush-compat"
}

ceph osd crush tree
ID  CLASS WEIGHT     (compat)  TYPE NAME          
 -1       3176.04785           root default       
 -7        316.52490 316.52490     host node0
  0   hdd    9.09560   9.09560         osd.0      
  4   hdd    9.09560   9.09560         osd.4      
  8   hdd    9.09560   9.09560         osd.8      
 10   hdd    9.09560   9.09560         osd.10     
 12   hdd    9.09560   9.09560         osd.12     
 16   hdd    9.09560   9.09560         osd.16     
 20   hdd    9.09560   9.09560         osd.20     
 21   hdd    9.09560   9.09560         osd.21     
 26   hdd    9.09560   9.09560         osd.26     
 29   hdd    9.09560   9.09560         osd.29     
 31   hdd    9.09560   9.09560         osd.31     
 35   hdd    9.09560   9.09560         osd.35     
 37   hdd    9.09560   9.09560         osd.37     
 44   hdd    9.09560   9.09560         osd.44     
 47   hdd    9.09560   9.09560         osd.47     
 56   hdd    9.09560   9.09560         osd.56     
 59   hdd    9.09560   9.09560         osd.59     
 65   hdd    9.09560   9.09560         osd.65     
 71   hdd    9.09560   9.09560         osd.71     
 77   hdd    9.09560   9.09560         osd.77     
 80   hdd    9.09560   9.09560         osd.80     
 83   hdd    9.09569   9.09569         osd.83     
 86   hdd    9.09560   9.09560         osd.86     
 88   hdd    9.09560   9.09560         osd.88     
 94   hdd   10.91409  10.91409         osd.94     
 95   hdd   10.91409  10.91409         osd.95     
 98   hdd   10.91409  10.91409         osd.98     
 99   hdd   10.91409  10.91409         osd.99     
238   hdd    9.09569   9.09569         osd.238    
239   hdd    9.09569   9.09569         osd.239    
240   hdd    9.09569   9.09569         osd.240    
241   hdd    9.09569   9.09569         osd.241    
242   hdd    9.09569   9.09569         osd.242    
243   hdd    9.09569   9.09569         osd.243    
 -3        316.52426 316.52426     host node1 
  1   hdd    9.09560   9.09560         osd.1      
  5   hdd    9.09560   9.09560         osd.5      
  6   hdd    9.09560   9.09560         osd.6      
 11   hdd    9.09560   9.09560         osd.11     
 13   hdd    9.09560   9.09560         osd.13     
 15   hdd    9.09560   9.09560         osd.15     
 19   hdd    9.09560   9.09560         osd.19     
 23   hdd    9.09560   9.09560         osd.23     
 25   hdd    9.09560   9.09560         osd.25     
 28   hdd    9.09560   9.09560         osd.28     
 32   hdd    9.09560   9.09560         osd.32     
 34   hdd    9.09560   9.09560         osd.34     
 38   hdd    9.09560   9.09560         osd.38     
 41   hdd    9.09560   9.09560         osd.41     
 43   hdd    9.09560   9.09560         osd.43     
 46   hdd    9.09560   9.09560         osd.46     
 49   hdd    9.09560   9.09560         osd.49     
 52   hdd    9.09560   9.09560         osd.52     
 55   hdd    9.09560   9.09560         osd.55     
 58   hdd    9.09560   9.09560         osd.58     
 61   hdd    9.09560   9.09560         osd.61     
 64   hdd    9.09560   9.09560         osd.64     
 67   hdd    9.09560   9.09560         osd.67     
 70   hdd    9.09560   9.09560         osd.70     
 73   hdd    9.09560   9.09560         osd.73     
 76   hdd    9.09560   9.09560         osd.76     
 79   hdd    9.09560   9.09560         osd.79     
 81   hdd    9.09560   9.09560         osd.81     
 85   hdd    9.09560   9.09560         osd.85     
 89   hdd    9.09560   9.09560         osd.89     
 90   hdd   10.91409  10.91409         osd.90     
 91   hdd   10.91409  10.91409         osd.91     
 96   hdd   10.91409  10.91409         osd.96     
 97   hdd   10.91409  10.91409         osd.97     
 -5        316.52426 316.52426     host node2 
  2   hdd    9.09560   9.09560         osd.2      
  3   hdd    9.09560   9.09560         osd.3      
  7   hdd    9.09560   9.09560         osd.7      
  9   hdd    9.09560   9.09560         osd.9      
 14   hdd    9.09560   9.09560         osd.14     
 17   hdd    9.09560   9.09560         osd.17     
 18   hdd    9.09560   9.09560         osd.18     
 22   hdd    9.09560   9.09560         osd.22     
 24   hdd    9.09560   9.09560         osd.24     
 27   hdd    9.09560   9.09560         osd.27     
 30   hdd    9.09560   9.09560         osd.30     
 33   hdd    9.09560   9.09560         osd.33     
 36   hdd    9.09560   9.09560         osd.36     
 39   hdd    9.09560   9.09560         osd.39     
 42   hdd    9.09560   9.09560         osd.42     
 45   hdd    9.09560   9.09560         osd.45     
 48   hdd    9.09560   9.09560         osd.48     
 51   hdd    9.09560   9.09560         osd.51     
 54   hdd    9.09560   9.09560         osd.54     
 57   hdd    9.09560   9.09560         osd.57     
 60   hdd    9.09560   9.09560         osd.60     
 63   hdd    9.09560   9.09560         osd.63     
 66   hdd    9.09560   9.09560         osd.66     
 69   hdd    9.09560   9.09560         osd.69     
 72   hdd    9.09560   9.09560         osd.72     
 75   hdd    9.09560   9.09560         osd.75     
 78   hdd    9.09560   9.09560         osd.78     
 82   hdd    9.09560   9.09560         osd.82     
 84   hdd    9.09560   9.09560         osd.84     
 87   hdd    9.09560   9.09560         osd.87     
 92   hdd   10.91409  10.91409         osd.92     
 93   hdd   10.91409  10.91409         osd.93     
100   hdd   10.91409  10.91409         osd.100    
101   hdd   10.91409  10.91409         osd.101    
-13        371.07864 371.07864     host node3 
102   hdd   10.91408  10.91408         osd.102    
103   hdd   10.91408  10.91408         osd.103    
104   hdd   10.91408  10.91408         osd.104    
105   hdd   10.91408  10.91408         osd.105    
106   hdd   10.91408  10.91408         osd.106    
107   hdd   10.91408  10.91408         osd.107    
108   hdd   10.91408  10.91408         osd.108    
109   hdd   10.91408  10.91408         osd.109    
110   hdd   10.91408  10.91408         osd.110    
111   hdd   10.91408  10.91408         osd.111    
112   hdd   10.91408  10.91408         osd.112    
113   hdd   10.91408  10.91408         osd.113    
114   hdd   10.91408  10.91408         osd.114    
115   hdd   10.91408  10.91408         osd.115    
116   hdd   10.91408  10.91408         osd.116    
117   hdd   10.91408  10.91408         osd.117    
118   hdd   10.91408  10.91408         osd.118    
119   hdd   10.91408  10.91408         osd.119    
120   hdd   10.91408  10.91408         osd.120    
121   hdd   10.91408  10.91408         osd.121    
122   hdd   10.91408  10.91408         osd.122    
123   hdd   10.91408  10.91408         osd.123    
124   hdd   10.91408  10.91408         osd.124    
125   hdd   10.91408  10.91408         osd.125    
126   hdd   10.91408  10.91408         osd.126    
127   hdd   10.91408  10.91408         osd.127    
128   hdd   10.91408  10.91408         osd.128    
129   hdd   10.91408  10.91408         osd.129    
130   hdd   10.91408  10.91408         osd.130    
131   hdd   10.91408  10.91408         osd.131    
132   hdd   10.91408  10.91408         osd.132    
133   hdd   10.91408  10.91408         osd.133    
134   hdd   10.91408  10.91408         osd.134    
135   hdd   10.91408  10.91408         osd.135    
-16        371.07916 371.07916     host node4 
136   hdd   10.91409  10.91409         osd.136    
137   hdd   10.91409  10.91409         osd.137    
138   hdd   10.91409  10.91409         osd.138    
139   hdd   10.91409  10.91409         osd.139    
140   hdd   10.91409  10.91409         osd.140    
141   hdd   10.91409  10.91409         osd.141    
142   hdd   10.91409  10.91409         osd.142    
143   hdd   10.91409  10.91409         osd.143    
144   hdd   10.91409  10.91409         osd.144    
145   hdd   10.91409  10.91409         osd.145    
146   hdd   10.91409  10.91409         osd.146    
147   hdd   10.91409  10.91409         osd.147    
148   hdd   10.91409  10.91409         osd.148    
149   hdd   10.91409  10.91409         osd.149    
150   hdd   10.91409  10.91409         osd.150    
151   hdd   10.91409  10.91409         osd.151    
152   hdd   10.91409  10.91409         osd.152    
153   hdd   10.91409  10.91409         osd.153    
154   hdd   10.91409  10.91409         osd.154    
155   hdd   10.91409  10.91409         osd.155    
156   hdd   10.91409  10.91409         osd.156    
157   hdd   10.91409  10.91409         osd.157    
158   hdd   10.91409  10.91409         osd.158    
159   hdd   10.91409  10.91409         osd.159    
160   hdd   10.91409  10.91409         osd.160    
161   hdd   10.91409  10.91409         osd.161    
162   hdd   10.91409  10.91409         osd.162    
163   hdd   10.91409  10.91409         osd.163    
164   hdd   10.91409  10.91409         osd.164    
165   hdd   10.91409  10.91409         osd.165    
166   hdd   10.91409  10.91409         osd.166    
167   hdd   10.91409  10.91409         osd.167    
168   hdd   10.91409  10.91409         osd.168    
169   hdd   10.91409  10.91409         osd.169    
-19        371.07916 371.07916     host node5 
170   hdd   10.91409  10.91409         osd.170    
171   hdd   10.91409  10.91409         osd.171    
173   hdd   10.91409  10.91409         osd.173    
174   hdd   10.91409  10.91409         osd.174    
175   hdd   10.91409  10.91409         osd.175    
176   hdd   10.91409  10.91409         osd.176    
177   hdd   10.91409  10.91409         osd.177    
178   hdd   10.91409  10.91409         osd.178    
179   hdd   10.91409  10.91409         osd.179    
180   hdd   10.91409  10.91409         osd.180    
181   hdd   10.91409  10.91409         osd.181    
182   hdd   10.91409  10.91409         osd.182    
183   hdd   10.91409  10.91409         osd.183    
184   hdd   10.91409  10.91409         osd.184    
185   hdd   10.91409  10.91409         osd.185    
186   hdd   10.91409  10.91409         osd.186    
187   hdd   10.91409  10.91409         osd.187    
188   hdd   10.91409  10.91409         osd.188    
189   hdd   10.91409  10.91409         osd.189    
190   hdd   10.91409  10.91409         osd.190    
191   hdd   10.91409  10.91409         osd.191    
192   hdd   10.91409  10.91409         osd.192    
193   hdd   10.91409  10.91409         osd.193    
194   hdd   10.91409  10.91409         osd.194    
195   hdd   10.91409  10.91409         osd.195    
196   hdd   10.91409  10.91409         osd.196    
197   hdd   10.91409  10.91409         osd.197    
198   hdd   10.91409  10.91409         osd.198    
199   hdd   10.91409  10.91409         osd.199    
200   hdd   10.91409  10.91409         osd.200    
201   hdd   10.91409  10.91409         osd.201    
202   hdd   10.91409  10.91409         osd.202    
203   hdd   10.91409  10.91409         osd.203    
237   hdd   10.91409  10.91409         osd.237    
-22        371.07916 371.07916     host node6 
172   hdd   10.91409  10.91409         osd.172    
204   hdd   10.91409  10.91409         osd.204    
205   hdd   10.91409  10.91409         osd.205    
206   hdd   10.91409  10.91409         osd.206    
207   hdd   10.91409  10.91409         osd.207    
208   hdd   10.91409  10.91409         osd.208    
209   hdd   10.91409  10.91409         osd.209    
210   hdd   10.91409  10.91409         osd.210    
211   hdd   10.91409  10.91409         osd.211    
212   hdd   10.91409  10.91409         osd.212    
213   hdd   10.91409  10.91409         osd.213    
214   hdd   10.91409  10.91409         osd.214    
215   hdd   10.91409  10.91409         osd.215    
216   hdd   10.91409  10.91409         osd.216    
217   hdd   10.91409  10.91409         osd.217    
218   hdd   10.91409  10.91409         osd.218    
219   hdd   10.91409  10.91409         osd.219    
220   hdd   10.91409  10.91409         osd.220    
221   hdd   10.91409  10.91409         osd.221    
222   hdd   10.91409  10.91409         osd.222    
223   hdd   10.91409  10.91409         osd.223    
224   hdd   10.91409  10.91409         osd.224    
225   hdd   10.91409  10.91409         osd.225    
226   hdd   10.91409  10.91409         osd.226    
227   hdd   10.91409  10.91409         osd.227    
228   hdd   10.91409  10.91409         osd.228    
229   hdd   10.91409  10.91409         osd.229    
230   hdd   10.91409  10.91409         osd.230    
231   hdd   10.91409  10.91409         osd.231    
232   hdd   10.91409  10.91409         osd.232    
233   hdd   10.91409  10.91409         osd.233    
234   hdd   10.91409  10.91409         osd.234    
235   hdd   10.91409  10.91409         osd.235    
236   hdd   10.91409  10.91409         osd.236    
-28        371.07916 371.07916     host node7 
 50   hdd   10.91409  10.91409         osd.50     
 53   hdd   10.91409  10.91409         osd.53     
 62   hdd   10.91409  10.91409         osd.62     
 68   hdd   10.91409  10.91409         osd.68     
 74   hdd   10.91409  10.91409         osd.74     
244   hdd   10.91409  10.91409         osd.244    
245   hdd   10.91409  10.91409         osd.245    
246   hdd   10.91409  10.91409         osd.246    
247   hdd   10.91409  10.91409         osd.247    
248   hdd   10.91409  10.91409         osd.248    
249   hdd   10.91409  10.91409         osd.249    
250   hdd   10.91409  10.91409         osd.250    
251   hdd   10.91409  10.91409         osd.251    
252   hdd   10.91409  10.91409         osd.252    
253   hdd   10.91409  10.91409         osd.253    
254   hdd   10.91409  10.91409         osd.254    
255   hdd   10.91409  10.91409         osd.255    
256   hdd   10.91409  10.91409         osd.256    
257   hdd   10.91409  10.91409         osd.257    
258   hdd   10.91409  10.91409         osd.258    
259   hdd   10.91409  10.91409         osd.259    
260   hdd   10.91409  10.91409         osd.260    
261   hdd   10.91409  10.91409         osd.261    
262   hdd   10.91409  10.91409         osd.262    
263   hdd   10.91409  10.91409         osd.263    
264   hdd   10.91409  10.91409         osd.264    
265   hdd   10.91409  10.91409         osd.265    
266   hdd   10.91409  10.91409         osd.266    
267   hdd   10.91409  10.91409         osd.267    
268   hdd   10.91409  10.91409         osd.268    
269   hdd   10.91409  10.91409         osd.269    
270   hdd   10.91409  10.91409         osd.270    
271   hdd   10.91409  10.91409         osd.271    
272   hdd   10.91409  10.91409         osd.272    
-25        371.07916 371.07916     host node8 
 40   hdd   10.91409  10.91409         osd.40     
273   hdd   10.91409  10.91409         osd.273    
274   hdd   10.91409  10.91409         osd.274    
275   hdd   10.91409  10.91409         osd.275    
276   hdd   10.91409  10.91409         osd.276    
277   hdd   10.91409  10.91409         osd.277    
278   hdd   10.91409  10.91409         osd.278    
279   hdd   10.91409  10.91409         osd.279    
280   hdd   10.91409  10.91409         osd.280    
281   hdd   10.91409  10.91409         osd.281    
282   hdd   10.91409  10.91409         osd.282    
283   hdd   10.91409  10.91409         osd.283    
284   hdd   10.91409  10.91409         osd.284    
285   hdd   10.91409  10.91409         osd.285    
286   hdd   10.91409  10.91409         osd.286    
287   hdd   10.91409  10.91409         osd.287    
288   hdd   10.91409  10.91409         osd.288    
289   hdd   10.91409  10.91409         osd.289    
290   hdd   10.91409  10.91409         osd.290    
291   hdd   10.91409  10.91409         osd.291    
292   hdd   10.91409  10.91409         osd.292    
293   hdd   10.91409  10.91409         osd.293    
294   hdd   10.91409  10.91409         osd.294    
295   hdd   10.91409  10.91409         osd.295    
296   hdd   10.91409  10.91409         osd.296    
297   hdd   10.91409  10.91409         osd.297    
298   hdd   10.91409  10.91409         osd.298    
299   hdd   10.91409  10.91409         osd.299    
300   hdd   10.91409  10.91409         osd.300    
301   hdd   10.91409  10.91409         osd.301    
302   hdd   10.91409  10.91409         osd.302    
303   hdd   10.91409  10.91409         osd.303    
304   hdd   10.91409  10.91409         osd.304    
305   hdd   10.91409  10.91409         osd.305

bye 
wolfgang
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux