On Tue, Apr 30, 2019 at 9:01 PM Igor Podlesny <ceph-user@xxxxxxxx> wrote: > > On Wed, 1 May 2019 at 01:26, Igor Podlesny <ceph-user@xxxxxxxx> wrote: > > On Wed, 1 May 2019 at 01:01, Dan van der Ster <dan@xxxxxxxxxxxxxx> wrote: > > >> > The upmap balancer in v12.2.12 works really well... Perfectly uniform on our clusters. > > >> > > >> mode upmap ? > > > > > > yes, mgr balancer, mode upmap. > > Also -- do your CEPHs have single root hierarchy pools (like > "default"), or there're some pools that use non-default ones? > > Looking through docs I didn't find a way to narrow balancer's scope > down to specific pool(s), although personally I would prefer it to > operate on a small set of them. > We have a mix of both single and dual root hierarchies -- the upmap balancer works for all. (E.g. this works: pool A with 3 replicas in root A, pool B with 3 replicas in root B. However if you have a cluster with two roots, and a pool that does something complex like put 2 replicas in root A and 1 replica in root B -- I haven't tested that recently). In luminous and mimic there isn't a way to scope the auto balancing down to limited pools. In practice that doesn't really matter, because of how it works, roughly: while true: select a random pool get the pg distribution for that pool create upmaps (or remove existing upmaps) to balance the pgs for that pool sleep 60s Eventually it attacks all pools and gets them fully balanced. (It anyway spends most of the time balancing the pools that matter, because the ones that don't have data get "balanced" quickly). If you absolutely must limit the pools, you have to script something to loop on `ceph balancer optimize myplan <mypool>; ceph balancer exec myplan` Something to reiterate: v12.2.12 has the latest upmap balancing heuristics, which are miles better than 12.2.11. (Big thanks to Xie Xingguo who worked hard to get this right!!!) Mimic v13.2.5 doesn't have those fixes (maybe in the pipeline for 13.2.6?) and I haven't checked Nautilus. If you're on mimic, then it's upmap balancer heuristics are better than nothing, but it might be imperfect or not work in certain cases (e.g. multi-root). -- Dan > -- > End of message. Next message? _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |