crush optimize does not work

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hello,

while trying to optimize a ceph cluster running jewel i get the
following output:
2017-10-26 10:43:27,615 argv = optimize --crushmap
/home/spriebe/ceph.report --out-path /home/spriebe/optimized.crush
--pool 5 --pool=5 --choose-args=5 --replication-count=3 --pg-num=4096
--pgp-num=4096 --rule=data --out-version=j --no-positions
2017-10-26 10:43:27,646 root optimizing
2017-10-26 10:43:29,329 root already optimized
2017-10-26 10:43:29,337 cloud1-1475 optimizing
2017-10-26 10:43:29,348 cloud1-1474 optimizing
2017-10-26 10:43:29,353 cloud1-1473 optimizing
2017-10-26 10:43:29,361 cloud1-1467 optimizing
2017-10-26 10:43:30,118 cloud1-1473 already optimized
2017-10-26 10:43:30,126 cloud1-1472 optimizing
2017-10-26 10:43:30,177 cloud1-1474 already optimized
2017-10-26 10:43:30,178 cloud1-1467 already optimized
2017-10-26 10:43:30,185 cloud1-1471 optimizing
2017-10-26 10:43:30,193 cloud1-1470 optimizing
2017-10-26 10:43:30,301 cloud1-1475 already optimized
2017-10-26 10:43:30,310 cloud1-1469 optimizing
2017-10-26 10:43:30,855 cloud1-1472 already optimized
2017-10-26 10:43:30,864 cloud1-1468 optimizing
2017-10-26 10:43:31,020 cloud1-1470 already optimized
2017-10-26 10:43:31,075 cloud1-1471 already optimized
2017-10-26 10:43:31,079 cloud1-1469 already optimized
2017-10-26 10:43:31,460 cloud1-1468 already optimized


But this one is heavily inbalanced if you look at the AVAIL GB.

ID WEIGHT  REWEIGHT SIZE   USE    AVAIL  %USE  VAR  PGS
31 1.00000  1.00000   931G   756G   174G 81.24 1.24 204
32 1.00000  1.00000   931G   750G   180G 80.58 1.23 202
33 1.00000  1.00000   931G   774G   157G 83.13 1.27 209
34 1.00000  1.00000   931G   743G   187G 79.89 1.22 200
48 1.00000  1.00000   931G   780G   150G 83.79 1.28 210
49 1.00000  1.00000   931G   778G   152G 83.64 1.27 209
50 0.79999  1.00000   819G   631G   188G 77.00 1.17 170
21 1.00000  0.95001   931G   729G   201G 78.32 1.19 197
22 1.00000  1.00000   931G   756G   174G 81.28 1.24 204
23 1.00000  1.00000   931G   778G   152G 83.57 1.27 210
24 1.00000  0.95001   931G   760G   171G 81.63 1.24 205
46 1.00000  1.00000   931G   756G   174G 81.30 1.24 203
47 1.00000  1.00000   931G   768G   162G 82.57 1.26 207
55 0.79999  1.00000   819G   602G   217G 73.46 1.12 162
11 1.00000  1.00000   931G   628G   302G 67.47 1.03 169
12 1.00000  1.00000   931G   630G   300G 67.68 1.03 170
13 1.00000  1.00000   931G   629G   301G 67.63 1.03 170
14 1.00000  1.00000   931G   644G   286G 69.20 1.05 173
45 1.00000  1.00000   931G   635G   295G 68.23 1.04 171
56 0.79999  1.00000   819G   509G   309G 62.18 0.95 137
40 1.72670  1.00000  1768G   774G   994G 43.78 0.67 209
51 1.00000  0.95001   931G   784G   146G 84.26 1.28 212
52 1.00000  1.00000   931G   768G   162G 82.52 1.26 207
53 1.00000  1.00000   931G   752G   178G 80.78 1.23 203
54 1.00000  1.00000   931G   750G   180G 80.58 1.23 202
38 1.00000  1.00000   931G   768G   162G 82.58 1.26 207
39 1.00000  1.00000   931G   768G   162G 82.51 1.26 207
57 0.79999  1.00000   819G   616G   203G 75.17 1.15 167
43 1.00000  1.00000   931G   636G   294G 68.41 1.04 171
44 1.00000  1.00000   931G   630G   300G 67.70 1.03 170
36 1.00000  1.00000   931G   626G   304G 67.33 1.03 170
37 1.00000  1.00000   931G   637G   293G 68.43 1.04 171
58 0.79999  1.00000   819G   508G   311G 62.03 0.95 137
41 0.87000  1.00000   888G   549G   339G 61.79 0.94 148
42 1.72670  1.00000  1768G   382G  1385G 21.65 0.33 103
65 1.72670  1.00000  1768G   383G  1384G 21.69 0.33 103
 1 1.00000  1.00000   931G   760G   170G 81.70 1.24 205
 2 1.00000  1.00000   931G   768G   162G 82.57 1.26 207
 3 1.00000  0.95001   931G   732G   198G 78.71 1.20 197
30 1.00000  1.00000   931G   771G   159G 82.84 1.26 209
35 1.00000  1.00000   931G   751G   179G 80.70 1.23 202
59 0.79999  1.00000   819G   599G   220G 73.08 1.11 162
 0 0.84999  1.00000   874G   662G   211G 75.81 1.16 179
 4 1.00000  1.00000   931G   633G   297G 68.07 1.04 171
 5 1.00000  1.00000   931G   632G   298G 67.94 1.04 171
60 0.79999  1.00000   819G   509G   310G 62.12 0.95 137
29 0.87000  1.00000   888G   553G   335G 62.29 0.95 149
 7 0.87000  1.00000   888G   554G   333G 62.44 0.95 149
28 0.87000  1.00000   888G   551G   336G 62.09 0.95 149
64 1.73000  1.00000  1768G  1088G   679G 61.57 0.94 293
 6 1.72670  1.00000  1768G   472G  1295G 26.71 0.41 127
66 1.72670  1.00000  1768G   475G  1292G 26.88 0.41 128
 8 1.00000  1.00000   931G   625G   305G 67.14 1.02 169
 9 1.00000  1.00000   931G   631G   299G 67.82 1.03 170
61 0.79999  1.00000   819G   507G   312G 61.90 0.94 137
15 0.87000  1.00000   888G   550G   338G 61.89 0.94 148
26 0.87000  1.00000   888G   551G   337G 62.04 0.95 149
27 0.84999  1.00000   874G   541G   333G 61.90 0.94 146
63 1.73000  1.00000  1768G  1101G   666G 62.31 0.95 297
10 1.72670  1.00000  1768G   483G  1284G 27.34 0.42 131
67 1.72670  1.00000  1768G   482G  1285G 27.31 0.42 130
16 1.00000  1.00000   931G   786G   144G 84.46 1.29 212
17 1.00000  1.00000   931G   783G   147G 84.12 1.28 211
18 1.00000  1.00000   931G   747G   183G 80.28 1.22 201
19 1.00000  1.00000   931G   758G   172G 81.50 1.24 205
20 1.00000  1.00000   931G   759G   171G 81.59 1.24 205
25 1.00000  1.00000   931G   766G   164G 82.35 1.25 207
62 0.79999  1.00000   819G   620G   199G 75.68 1.15 166
31 1.00000  1.00000   931G   756G   174G 81.24 1.24 204
32 1.01900  1.00000   931G   750G   180G 80.58 1.23 202
33 1.00000  1.00000   931G   774G   157G 83.13 1.27 209
34 0.92699  1.00000   931G   743G   187G 79.89 1.22 200
48 1.00000  1.00000   931G   780G   150G 83.79 1.28 210
49 1.02399  1.00000   931G   778G   152G 83.64 1.27 209
50 0.82999  1.00000   819G   631G   188G 77.00 1.17 170
21 1.12793  0.95001   931G   729G   201G 78.32 1.19 197
22 0.93199  1.00000   931G   756G   174G 81.28 1.24 204
23 0.94606  1.00000   931G   778G   152G 83.57 1.27 210
24 0.95599  0.95001   931G   760G   171G 81.63 1.24 205
46 1.01500  1.00000   931G   756G   174G 81.30 1.24 203
47 0.99500  1.00000   931G   768G   162G 82.57 1.26 207
55 0.82899  1.00000   819G   602G   217G 73.46 1.12 162
11 0.78575  1.00000   931G   628G   302G 67.47 1.03 169
12 0.89369  1.00000   931G   630G   300G 67.68 1.03 170
13 0.84279  1.00000   931G   629G   301G 67.63 1.03 170
14 0.77834  1.00000   931G   644G   286G 69.20 1.05 173
45 0.83331  1.00000   931G   635G   295G 68.23 1.04 171
56 0.74358  1.00000   819G   509G   309G 62.18 0.95 137
40 0.96149  1.00000  1768G   774G   994G 43.78 0.67 209
51 1.03899  0.95001   931G   784G   146G 84.26 1.28 212
52 0.98000  1.00000   931G   768G   162G 82.52 1.26 207
53 1.01900  1.00000   931G   752G   178G 80.78 1.23 203
54 0.98000  1.00000   931G   750G   180G 80.58 1.23 202
38 0.94199  1.00000   931G   768G   162G 82.58 1.26 207
39 1.03899  1.00000   931G   768G   162G 82.51 1.26 207
57 0.79999  1.00000   819G   616G   203G 75.17 1.15 167
43 0.77654  1.00000   931G   636G   294G 68.41 1.04 171
44 0.85941  1.00000   931G   630G   300G 67.70 1.03 170
36 0.85104  1.00000   931G   626G   304G 67.33 1.03 170
37 0.80946  1.00000   931G   637G   293G 68.43 1.04 171
58 0.67404  1.00000   819G   508G   311G 62.03 0.95 137
41 0.68245  1.00000   888G   549G   339G 61.79 0.94 148
42 0.49387  1.00000  1768G   382G  1385G 21.65 0.33 103
65 0.52313  1.00000  1768G   383G  1384G 21.69 0.33 103
 1 0.96100  1.00000   931G   760G   170G 81.70 1.24 205
 2 0.94199  1.00000   931G   768G   162G 82.57 1.26 207
 3 1.03499  0.95001   931G   732G   198G 78.71 1.20 197
30 1.00000  1.00000   931G   771G   159G 82.84 1.26 209
35 1.07399  1.00000   931G   751G   179G 80.70 1.23 202
59 0.87000  1.00000   819G   599G   220G 73.08 1.11 162
 0 0.76900  1.00000   874G   662G   211G 75.81 1.16 179
 4 0.73238  1.00000   931G   633G   297G 68.07 1.04 171
 5 0.88098  1.00000   931G   632G   298G 67.94 1.04 171
60 0.67899  1.00000   819G   509G   310G 62.12 0.95 137
29 0.72182  1.00000   888G   553G   335G 62.29 0.95 149
 7 0.71684  1.00000   888G   554G   333G 62.44 0.95 149
28 0.64395  1.00000   888G   551G   336G 62.09 0.95 149
64 1.46759  1.00000  1768G  1088G   679G 61.57 0.94 293
 6 0.59332  1.00000  1768G   472G  1295G 26.71 0.41 127
66 0.68506  1.00000  1768G   475G  1292G 26.88 0.41 128
 8 0.89154  1.00000   931G   625G   305G 67.14 1.02 169
 9 0.83815  1.00000   931G   631G   299G 67.82 1.03 170
61 0.64093  1.00000   819G   507G   312G 61.90 0.94 137
15 0.74059  1.00000   888G   550G   338G 61.89 0.94 148
26 0.65033  1.00000   888G   551G   337G 62.04 0.95 149
27 0.61040  1.00000   874G   541G   333G 61.90 0.94 146
63 1.44260  1.00000  1768G  1101G   666G 62.31 0.95 297
10 0.65889  1.00000  1768G   483G  1284G 27.34 0.42 131
67 0.60854  1.00000  1768G   482G  1285G 27.31 0.42 130
16 0.98000  1.00000   931G   786G   144G 84.46 1.29 212
17 1.11299  1.00000   931G   783G   147G 84.12 1.28 211
18 1.12399  1.00000   931G   747G   183G 80.28 1.22 201
19 0.87099  1.00000   931G   758G   172G 81.50 1.24 205
20 0.95099  1.00000   931G   759G   171G 81.59 1.24 205
25 1.04700  1.00000   931G   766G   164G 82.35 1.25 207
62 0.71399  1.00000   819G   620G   199G 75.68 1.15 166
              TOTAL 69475G 45596G 23878G 65.63
MIN/MAX VAR: 0.33/1.29  STDDEV: 16.64

crush tool is 1.0.35

Greets,
Stefan
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux