bluestore high latency in hdd and ssd mixed scene

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi all,

I tested the performance of bluestore in a mixed scene of hdd and ssd.
There are three nodes here, each node has 1 mon and 1 mgr 12 osd,
Each osd has 1.7T HDD for block, 150G SSD partition for db and wal, 10Gib bond.

system: CentOS Linux (3.10.0-862.3.2.el7.x86_64) 7 (Core)
ceph version 12.2.5 (cad919881333ac92274171586c827e01f554a70a) luminous (stable)

I tested that the performance is very poor and the delay is hug.
I found that _kv_sync_thread force_flush took a long time to block the kv commit.
so 4krandwrite 32dp IOPS=2469, BW=9878KiB/s lat 12955.49 us. 

I have some thinking about mixed scene of hdd and ssd:
Hdd fdatasync is slow,Can we avoid fdatasync on the wite path(_kv_sync_thread)?
batch sync to hdd in the background,For deferred_write can use wal for caching,
Is it possible to do a cache on ssd for simple write?  

Have we had a discussion about the built-in ssd cache for bluestore?--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html




[Index of Archives]     [CEPH Users]     [Ceph Large]     [Information on CEPH]     [Linux BTRFS]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]

  Powered by Linux