> > This is what I have when I query prometheus, most hdd's are still sata
> 5400rpm, there are also some ssd's. I also did not optimize cpu
> frequency settings. (forget about the instance=c03, that is just because
> the data comes from mgr c03, these drives are on different hosts)
> >
> > ceph_osd_apply_latency_ms
> >
> > ceph_osd_apply_latency_ms{ceph_daemon="osd.12", instance="c03",
> job="ceph"} 42
> > ...
> > ceph_osd_apply_latency_ms{ceph_daemon="osd.19", instance="c03",
> job="ceph"} 1
>
> I assume this looks somewhat normal, with a bit of variance due to
> access.
>
> > avg (ceph_osd_apply_latency_ms)
> > 9.333333333333336
>
> I see something similar, around 9ms average latency for HDD based osds,
> best case average around 3ms.
>
> > So I guess it is possible for you to get lower values on the lsi hba
>
> Can you let me know which exact model you have?
[~]# sas2flash -list
LSI Corporation SAS2 Flash Utility
Version 20.00.00.00 (2014.09.18)
Copyright (c) 2008-2014 LSI Corporation. All rights reserved
Adapter Selected is a LSI SAS: SAS2308_2(D1)
Controller Number : 0
Controller : SAS2308_2(D1)
PCI Address : 00:04:00:00
SAS Address : 500605b-0-05a6-c49e
NVDATA Version (Default) : 14.01.00.06
NVDATA Version (Persistent) : 14.01.00.06
Firmware Product ID : 0x2214 (IT)
Firmware Version : 20.00.07.00
NVDATA Vendor : LSI
NVDATA Product ID : SAS9207-8i
BIOS Version : 07.39.02.00
UEFI BSD Version : N/A
FCODE Version : N/A
Board Name : SAS9207-8i
Board Assembly : N/A
Board Tracer Number : N/A
>
> > Maybe you can tune read a head on the lsi with something like this.
> > echo 8192 > /sys/block/$line/queue/read_ahead_kb
> > echo 1024 > /sys/block/$line/queue/nr_requests
>
> I tried both of them, even going up to 16MB read ahead cache, but
> besides a short burst when changing the values, the average stays +/-
> the same on that host.
>
> I also checked cpu speed (same as the rest), io scheduler (using "none"
> really drives the disks crazy). What I observed is that the avq value in
> atop is lower than on the other servers, which are around 15. This
> server is more in the range 1-3.
>
> > Also check for pci-e 3 those have higher bus speeds.
>
> True, even though pci-e 2, x8 should be able to deliver 4 GB/s, if I am
> not mistaken.
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
