On Tuesday 11 Jun 2013, OS Engineering wrote: > Hi Jens, > > In continuation with our previous communication, we have carried out > performance comparison among EnhanceIO, bcache and dm-cache. > > We found that EnhanceIO provides better throughput on zipf workload (with > theta=1.2) in comparison to bcache and dm-cache for write through caches. > However, for write back caches, we found that dm-cache had best throughput > followed by EnhanceIO and then bcache. Dm-cache commits on-disk metadata > every time a REQ_SYNC or REQ_FUA bio is written. If no such requests are > made then it commits metadata once every second. If power is lost, it may > lose some recent writes. However, EnhanceIO and bcache do not acknowledge > IO completion until both IO and metadata hits the SSD. Hence, EnhanceIO > and bcache provide higher data integrity at a cost of performance. So it won't be fair to compare them with dm-cache in write-back mode since guarantees are different. I am sure if similar (SYNC/FUA enforcement for persistence) guarantees are implemented in bcache/EnhanceIO, they'll offer a much better performance. > > The fio config and setup information follows: > HDD : 100GB > SSD : 20GB > Mode : write through / write back > Cache block_size : 4KB for bcache, EnhanceIO and 256KB for dm-cache > > The other options have been left to their default values. > > Note: > 1) In case of dm-cache, we used 2 partitions of same SSD with 1GB partition > as metadata and 20GB partition as caching device. This has been done so as > to ensure a fair comparison as EnhanceIO and bcache do not have a separate > metadata device. > > 2) In order to ensure proper cache warm up, We have turned off sequential > bypass in bcache. This does not impact our performance results as they are > taken for random workload. > > For each test, we first performed a warm up run using the following fio > options: fio --direct=1 --size=90% --filesize=20G --blocksize=4k > --ioengine=libaio --rw=rw --rwmixread=100 --rwmixwrite=0 --iodepth=8 ... > > Then, we performed our actual run with the following fio options: > fio --direct=1 --size=100% --filesize=20G --blocksize=4k --ioengine=libaio > --rw=randrw --rwmixread=90 --rwmixwrite=10 --iodepth=8 --numjobs=4 > --random_distribution=zipf:1.2 ... Did you experiment a little with varying iodepth and numjobs? Is this the best combination you found out? The numbers below appear low considering a 90% hit ratio for EnhanceIO. SSD baseline performance shown below appears low. However it should not affect this comparison. All caching solutions are being subjected to the same ratio of HDD/SSD performance. > > ============================= Write Through =============================== > Type Read Latency(ms) Write Latency(ms) Read(MB/s) Write(MB/s) > =========================================================================== > EnhanceIO 1.58 16.53 32.91 3.65 > bcache 0.58 31.05 27.17 3.02 > dm-cache 0.24 27.45 31.05 3.44 EnhanceIO shows much higher read latency here. Is that because of READFILLs ? Write latency, read-write throughputs are good. > > ============================= Write Back ================================== > Type Read Latency(ms) Write Latency(ms) Read(MB/s) Write(MB/s) > =========================================================================== > EnhanceIO 0.34 4.98 138.72 15.40 > bcache 0.95 1.76 106.82 11.85 > dm-cache 0.58 0.55 193.76 21.52 Here EnhanceIO's read latency is better compared the other two. Write latency is larger than the other two. -Amit > > ============================ Base Line ==================================== > Type Read Latency(ms) Write Latency(ms) Read(MB/s) Write(MB/s) > =========================================================================== > HDD 6.22 27.23 13.51 1.49 > SSD 0.47 0.42 235.87 26.21 > > We have written scripts that aid in cache creation, deletion and > performance run for all these caching solutions. These scripts can be > found at: > https://github.com/stec-inc/EnhanceIO/tree/master/performance_test > > Thanks and Regards, > sTec Team > > PROPRIETARY-CONFIDENTIAL INFORMATION INCLUDED > > This electronic transmission, and any documents attached hereto, may > contain confidential, proprietary and/or legally privileged information. > The information is intended only for use by the recipient named above. If > you received this electronic message in error, please notify the sender > and delete the electronic message. Any disclosure, copying, distribution, > or use of the contents of information received in error is strictly > prohibited, and violators will be pursued legally. -- > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ -- dm-devel mailing list dm-devel@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/dm-devel
