Hi Vladimir, Yes, device caching is disabled. The ODSs in question use a > separate DB/WAL device on flash. Could that be the cause of > IOPS over-estimation? > 2000 IOPS definitely looks high for a HDD. The in-built osd bench tool writes to the backing device of the osd to determine the IOPS capacity. With DB/WAL configured, I wouldn't expect a wide variation. To confirm the IOPS measurement, you can run the following commands: $ ceph tell osd.N cache drop followed by, $ ceph tell osd.N bench 12288000 4096 4194304 100 Where N is the osd id. You can run the above sequence a few times to confirm if there are any wide variations and if so, it probably warrants more investigation into why that's the case. Also, may I know the following: - what kind of HDDs are you using in your environment? - Ceph version Is it better for me to manually set > osd_mclock_max_capacity_iops_hdd to a reasonable number or > will things perform OK as long as IOPS are over-estimated > consistently everywhere? > I think it's better to figure out why the IOPS are on the higher side first. -Sridhar <https://www.redhat.com> _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |