Thomas,
I have a master branch version of the code to test. The nautilus
backport https://github.com/ceph/ceph/pull/31956 should be the same.
Using your OSDMap, the code in master branch and some additional changes
to osdmaptool I was able to balance your cluster. The osdmaptool
changes simulate the mgr active balancer behavior. It took 302 rounds
with 10 maximum upmaps per crush rule set of pools per round. With the
default 1 minute sleeps inside the mgr it would take about 5 hours.
Obviously, recovery/backfill has to finish before the cluster settles
into the new configuration. It needed 3402 additional upmaps.
Because all pools for a given crush rule are balanced together you can
see that this is more balanced than Rich's configuration using Luminous.
The pool hdb_backup which has its own rule was the most difficult to
balance.
This balancer code is subject to change before final release of the next
Nautilus point release.
Final layout:
osd.0 pgs 33
osd.1 pgs 31
osd.2 pgs 34
osd.3 pgs 33
osd.4 pgs 31
osd.5 pgs 31
osd.6 pgs 32
osd.7 pgs 31
osd.8 pgs 264
osd.9 pgs 28
osd.10 pgs 29
osd.11 pgs 28
osd.12 pgs 263
osd.13 pgs 27
osd.14 pgs 27
osd.15 pgs 27
osd.16 pgs 262
osd.17 pgs 27
osd.18 pgs 27
osd.19 pgs 262
osd.20 pgs 28
osd.21 pgs 27
osd.22 pgs 28
osd.23 pgs 29
osd.24 pgs 29
osd.25 pgs 28
osd.26 pgs 263
osd.27 pgs 28
osd.28 pgs 28
osd.29 pgs 27
osd.30 pgs 262
osd.31 pgs 86
osd.32 pgs 86
osd.33 pgs 86
osd.34 pgs 86
osd.35 pgs 87
osd.36 pgs 86
osd.37 pgs 86
osd.38 pgs 86
osd.39 pgs 86
osd.40 pgs 86
osd.41 pgs 86
osd.42 pgs 49
osd.43 pgs 62
osd.44 pgs 62
osd.45 pgs 56
osd.46 pgs 49
osd.47 pgs 56
osd.48 pgs 56
osd.49 pgs 56
osd.50 pgs 49
osd.51 pgs 62
osd.52 pgs 56
osd.53 pgs 43
osd.54 pgs 55
osd.55 pgs 56
osd.56 pgs 49
osd.57 pgs 49
osd.58 pgs 49
osd.59 pgs 86
osd.60 pgs 86
osd.61 pgs 86
osd.62 pgs 86
osd.63 pgs 86
osd.64 pgs 87
osd.65 pgs 86
osd.66 pgs 87
osd.67 pgs 87
osd.68 pgs 87
osd.69 pgs 86
osd.70 pgs 86
osd.71 pgs 86
osd.72 pgs 86
osd.73 pgs 87
osd.74 pgs 87
osd.75 pgs 86
osd.76 pgs 49
osd.77 pgs 49
osd.78 pgs 55
osd.79 pgs 49
osd.80 pgs 62
osd.81 pgs 62
osd.82 pgs 62
osd.83 pgs 49
osd.84 pgs 56
osd.85 pgs 61
osd.86 pgs 49
osd.87 pgs 61
osd.88 pgs 62
osd.89 pgs 43
osd.90 pgs 49
osd.91 pgs 55
osd.92 pgs 62
osd.93 pgs 61
osd.94 pgs 55
osd.95 pgs 56
osd.96 pgs 49
osd.97 pgs 43
osd.98 pgs 49
osd.99 pgs 56
osd.100 pgs 43
osd.101 pgs 49
osd.102 pgs 43
osd.103 pgs 55
osd.104 pgs 43
osd.105 pgs 43
osd.106 pgs 62
osd.107 pgs 49
osd.108 pgs 62
osd.109 pgs 62
osd.110 pgs 62
osd.111 pgs 62
osd.112 pgs 49
osd.113 pgs 49
osd.114 pgs 49
osd.115 pgs 49
osd.116 pgs 62
osd.117 pgs 56
osd.118 pgs 55
osd.119 pgs 49
osd.120 pgs 43
osd.121 pgs 55
osd.122 pgs 31
osd.123 pgs 56
osd.124 pgs 62
osd.125 pgs 62
osd.126 pgs 62
osd.127 pgs 49
osd.128 pgs 49
osd.129 pgs 56
osd.130 pgs 61
osd.131 pgs 55
osd.132 pgs 55
osd.133 pgs 55
osd.134 pgs 49
osd.135 pgs 55
osd.136 pgs 55
osd.137 pgs 43
osd.138 pgs 55
osd.139 pgs 55
osd.140 pgs 49
osd.141 pgs 43
osd.142 pgs 55
osd.143 pgs 55
osd.144 pgs 55
osd.145 pgs 55
osd.146 pgs 55
osd.147 pgs 62
osd.148 pgs 55
osd.149 pgs 62
osd.150 pgs 49
osd.151 pgs 55
osd.152 pgs 55
osd.153 pgs 49
osd.154 pgs 49
osd.155 pgs 55
osd.156 pgs 43
osd.157 pgs 49
osd.157 pgs 49
osd.158 pgs 62
osd.159 pgs 49
osd.160 pgs 55
osd.161 pgs 49
osd.162 pgs 62
osd.163 pgs 49
osd.164 pgs 55
osd.165 pgs 62
osd.166 pgs 49
osd.167 pgs 43
osd.168 pgs 49
osd.169 pgs 61
osd.170 pgs 62
osd.171 pgs 55
osd.172 pgs 87
osd.173 pgs 87
osd.174 pgs 87
osd.175 pgs 87
osd.176 pgs 87
osd.177 pgs 86
osd.178 pgs 86
osd.179 pgs 87
osd.180 pgs 86
osd.181 pgs 86
osd.182 pgs 87
osd.183 pgs 87
osd.184 pgs 86
osd.185 pgs 86
osd.186 pgs 87
osd.187 pgs 86
osd.188 pgs 87
osd.189 pgs 43
osd.190 pgs 49
osd.191 pgs 55
osd.192 pgs 62
osd.193 pgs 55
osd.194 pgs 43
osd.195 pgs 62
osd.196 pgs 43
osd.197 pgs 49
osd.198 pgs 62
osd.199 pgs 55
osd.200 pgs 37
osd.201 pgs 55
osd.202 pgs 49
osd.203 pgs 49
osd.204 pgs 62
osd.205 pgs 62
osd.206 pgs 55
osd.207 pgs 49
osd.208 pgs 43
osd.209 pgs 43
osd.210 pgs 43
osd.211 pgs 49
osd.212 pgs 49
osd.213 pgs 49
osd.214 pgs 55
osd.215 pgs 43
osd.216 pgs 49
osd.217 pgs 49
osd.218 pgs 55
osd.219 pgs 62
osd.220 pgs 49
osd.221 pgs 55
osd.222 pgs 49
osd.223 pgs 55
osd.224 pgs 49
osd.225 pgs 55
osd.226 pgs 49
osd.227 pgs 55
osd.228 pgs 62
osd.229 pgs 62
osd.230 pgs 37
osd.231 pgs 55
osd.232 pgs 62
osd.233 pgs 55
osd.234 pgs 55
osd.235 pgs 62
osd.236 pgs 55
osd.237 pgs 62
osd.238 pgs 61
osd.239 pgs 37
osd.240 pgs 62
osd.241 pgs 62
osd.242 pgs 61
osd.243 pgs 62
osd.244 pgs 62
osd.245 pgs 62
osd.246 pgs 62
osd.247 pgs 55
osd.248 pgs 62
osd.249 pgs 49
osd.250 pgs 61
osd.251 pgs 49
osd.252 pgs 43
osd.253 pgs 55
osd.254 pgs 49
osd.255 pgs 55
osd.256 pgs 43
osd.257 pgs 55
osd.258 pgs 62
osd.259 pgs 61
osd.260 pgs 55
osd.261 pgs 43
osd.262 pgs 43
osd.263 pgs 49
osd.264 pgs 49
osd.265 pgs 62
osd.266 pgs 49
osd.267 pgs 62
osd.268 pgs 87
osd.269 pgs 86
osd.270 pgs 87
osd.271 pgs 86
osd.272 pgs 270
osd.273 pgs 271
osd.274 pgs 270
osd.275 pgs 270
osd.276 pgs 270
osd.277 pgs 270
osd.278 pgs 270
osd.279 pgs 270
osd.280 pgs 270
osd.281 pgs 270
osd.282 pgs 270
osd.283 pgs 271
osd.284 pgs 270
osd.285 pgs 271
osd.286 pgs 270
osd.287 pgs 270
osd.288 pgs 270
osd.289 pgs 270
osd.290 pgs 270
osd.291 pgs 270
osd.292 pgs 270
osd.293 pgs 270
osd.294 pgs 270
osd.295 pgs 270
osd.296 pgs 270
osd.297 pgs 270
osd.298 pgs 270
osd.299 pgs 270
osd.300 pgs 270
osd.301 pgs 270
osd.302 pgs 270
osd.303 pgs 270
osd.304 pgs 270
osd.305 pgs 270
osd.306 pgs 270
osd.307 pgs 270
osd.308 pgs 270
osd.309 pgs 270
osd.310 pgs 270
osd.311 pgs 270
osd.312 pgs 270
osd.313 pgs 270
osd.314 pgs 270
osd.315 pgs 270
osd.316 pgs 270
osd.317 pgs 270
osd.318 pgs 270
osd.319 pgs 270
osd.320 pgs 271
osd.321 pgs 271
osd.322 pgs 271
osd.323 pgs 271
osd.324 pgs 271
osd.325 pgs 270
osd.326 pgs 270
osd.327 pgs 270
osd.328 pgs 270
osd.329 pgs 270
osd.330 pgs 270
osd.331 pgs 270
osd.332 pgs 271
osd.333 pgs 270
osd.334 pgs 271
osd.335 pgs 270
osd.336 pgs 270
osd.337 pgs 271
osd.338 pgs 271
osd.339 pgs 270
osd.340 pgs 270
osd.341 pgs 270
osd.342 pgs 270
osd.343 pgs 270
osd.344 pgs 270
osd.345 pgs 271
osd.346 pgs 270
osd.347 pgs 271
osd.348 pgs 270
osd.349 pgs 270
osd.350 pgs 270
osd.351 pgs 270
osd.352 pgs 270
osd.353 pgs 270
osd.354 pgs 270
osd.355 pgs 270
osd.356 pgs 270
osd.357 pgs 270
osd.358 pgs 270
osd.359 pgs 270
osd.360 pgs 270
osd.361 pgs 270
osd.362 pgs 270
osd.363 pgs 270
osd.364 pgs 270
osd.365 pgs 270
osd.366 pgs 270
osd.367 pgs 270
osd.368 pgs 86
osd.369 pgs 86
osd.370 pgs 87
osd.371 pgs 270
osd.372 pgs 270
osd.373 pgs 270
osd.374 pgs 270
osd.375 pgs 270
osd.376 pgs 270
osd.377 pgs 270
osd.378 pgs 270
osd.379 pgs 270
osd.380 pgs 270
osd.381 pgs 270
osd.382 pgs 270
osd.383 pgs 270
osd.384 pgs 271
osd.385 pgs 270
osd.386 pgs 270
osd.387 pgs 270
osd.388 pgs 270
osd.389 pgs 270
osd.390 pgs 270
osd.391 pgs 270
osd.392 pgs 270
osd.393 pgs 270
osd.394 pgs 270
osd.395 pgs 270
osd.396 pgs 270
osd.397 pgs 270
osd.398 pgs 270
osd.399 pgs 270
osd.400 pgs 270
osd.401 pgs 270
osd.402 pgs 270
osd.403 pgs 270
osd.404 pgs 270
osd.405 pgs 270
osd.406 pgs 271
osd.407 pgs 270
osd.408 pgs 270
osd.409 pgs 270
osd.410 pgs 270
osd.411 pgs 270
osd.412 pgs 270
osd.413 pgs 270
osd.414 pgs 270
osd.415 pgs 270
osd.416 pgs 270
osd.417 pgs 270
osd.418 pgs 270
osd.419 pgs 87
osd.420 pgs 87
osd.421 pgs 86
osd.422 pgs 86
osd.423 pgs 86
osd.424 pgs 86
osd.425 pgs 86
osd.426 pgs 86
osd.427 pgs 86
osd.428 pgs 86
osd.429 pgs 86
osd.430 pgs 86
osd.431 pgs 86
osd.432 pgs 86
osd.433 pgs 86
osd.434 pgs 86
osd.435 pgs 58
osd.436 pgs 58
osd.437 pgs 58
osd.438 pgs 58
osd.439 pgs 58
osd.440 pgs 58
osd.441 pgs 58
David
On 12/11/19 6:12 AM, Thomas Schneider wrote:
Hello David,
I'm experiencing issues with OSD balancing, too.
My ceph cluster is running on release
ceph version 14.2.4.1 (596a387fb278758406deabf997735a1f706660c9)
nautilus (stable)
Would you be able to test (the latest code) on my OSDmap and verify if
balancing would work?
I have attached it to this email.
Regards
Thomas
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
