That's true, but we do intend to scale this cluster as necessary. Some new nodes are already being prepared, the only thing I'm really worried about is further expansion: we're now getting 3+ months of lead time on some components and vendors suggest that it will get worse :-( /Z On Wed, Oct 6, 2021 at 11:16 AM Janne Johansson <icepic.dz@xxxxxxxxx> wrote: > Den ons 6 okt. 2021 kl 10:11 skrev Zakhar Kirpichenko <zakhar@xxxxxxxxx>: > > > > I've initially disabled power-saving features, which nicely improved the > > network latency. > > > > Btw, the first interesting find: I enabled 'rbd_balance_parent_reads' on > > the clients, and single-thread reads now scale much better, I routinely > get > > similar readings from a single disk doing 4k reads with 1 thread: > > > > Run status group 0 (all jobs): > > READ: bw=323MiB/s (339MB/s), 323MiB/s-323MiB/s (339MB/s-339MB/s), > > io=18.9GiB (20.3GB), run=60001-60001msec > > Disk stats (read/write): > > vdc: ios=77451/0, merge=0/0, ticks=80269/0, in_queue=19964, util=97.94% > > > > No more 50 MB/s reads, yay! :-) > > > > The option, which I found in Redhat's docs, suggests that "Ceph typically > > reads objects from the primary OSD. Since reads are immutable, you may > > enable this feature to balance parent reads between the primary OSD and > the > > replicas." > > Do mind that while this might be very good for benchmarks while the > cluster is somewhat quiet, it might be worse-or-not-better when you > have 100 clients pushing IO requests to the cluster, since by then the > sum of IOs is already spread over all the OSDs, so there might be less > or no "idle" OSDs to request from. > > But it is nice if you can test this before and after real load hits > the cluster, for the science. =) > > > -- > May the most significant bit of your life be positive. > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |