On 11/28/19 12:56 PM, David Majchrzak, ODERLAND Webbhotell AB wrote: > Hi! > > We've deployed a new flash only ceph cluster running Nautilus and I'm > currently looking at any tunables we should set to get the most out of > our NVMe SSDs. > > I've been looking a bit at the options from the blog post here: > > https://ceph.io/community/bluestore-default-vs-tuned-performance-comparison/ > > with the conf here: > https://gist.github.com/likid0/1b52631ff5d0d649a22a3f30106ccea7 > > However some of them, like checksumming, is for testing speed only but > not really applicable in a real life scenario with critical data. > > Should we stick with defaults or is there anything that could help? > > We have 256GB of RAM on each OSD host, 8 OSD hosts with 10 SSDs on > each. 2 osd daemons on each SSD. Raise ssd bluestore cache to 8GB? > > Workload is about 50/50 r/w ops running qemu VMs through librbd. So > mixed block size. Pin the C-State of your CPUs to 1 and disable powersaving. That can reduce the latency vastly. Testing with rados bench -t 1 -b 4096 -o 4096 you should be able to get to a 0.8ms write latency with 3x replication. > > 3 replicas. > > Appreciate any advice! > > Kind Regards, > _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |