Hi Dan,
Great! Thanks for the feedback. Much appreciated.
For completion here is my ansible role:
- name: Increase aio-max-nr for bluestore
sysctl:
name: fs.aio-max-nr
value: 1048576
sysctl_file: /etc/sysctl.d/ceph-tuning.conf
sysctl_set: yes
Cheers,
Tom
On Wed, Aug 30, 2017 at 10:49 AM, Dan van der Ster <dan@xxxxxxxxxxxxxx> wrote:
Hi Thomas,
Yes we set it to a million.
>From our puppet manifest:
# need to increase aio-max-nr to allow many bluestore devs
sysctl { 'fs.aio-max-nr': val => '1048576' }
Cheers, Dan
> ______________________________
On Aug 30, 2017 9:53 AM, "Thomas Bennett" <thomas@xxxxxxxxx> wrote:
>
> Hi,
>
> I've been testing out Luminous and I've noticed that at some point the number of osds per nodes was limited by aio-max-nr. By default its set to 65536 in Ubuntu 16.04
>
> Has anyone else experienced this issue?
>
> fs.aio-nr currently sitting at 196608 with 48 osds.
>
> I have 48 osd's per node so I've added fs.aio-max-nr = 262144 in my /etc/systctl.d/ceph-tuning.conf file.
>
> Cheers,
> Tom
>
_________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph. com
>
Thomas Bennett
SKA South Africa
Science Processing Team
Office: +27 21 5067341
Mobile: +27 79 5237105
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
