Thanks Richard, That sounds impressive, especially the around 30% hit ratio. That would be ideal for me, but we were only getting single digit results during my trials. I think around 5% was the figure if I remember correctly. However, most of our vms were created a bit chaotically (not using predefined vm templates), which could be the reason for the low cache hit rate. If most of your vms are created from templates, you should get a far better cache performance figures I think. But the main factor is the data type, which is very random for our case as we have different applications and services. Would be interested to learn how you are achieving your 30-35% hit ratio on 700 vms. Cheers ----- Original Message ----- > From: "Richard Bade" <hitrich@xxxxxxxxx> > To: "ceph-users" <ceph-users@xxxxxxxxxxxxxx> > Sent: Thursday, 28 June, 2018 23:56:59 > Subject: Re: Luminous Bluestore performance, bcache > Hi Andrei, > These are good questions. We have another cluster with filestore and > bcache but for this particular one I was interested in testing out > bluestore. So I have used bluestore both with and without bcache. > For my synthetic load on the vm's I'm using this fio command: > fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 > --name=test --filename=test --bs=4k --iodepth=64 --size=4G > --readwrite=randwrite --rate_iops=50 > > Currently on bluestore with my synthetic load I'm getting 7% hit ratio > (cat /sys/block/bcache*/bcache/stats_total/cache_hit_ratio) > On our filestore cluster with ~700 vm's of varied workload we're > geting about 30-35% hit ratio. > In the hourly hit ratio I have as high as 50% on some osd's in our > filestore cluster. Only 25% on my synthetic load on bluestore so far, > but I hadn't actually been checking this stat until now. > > I hope that helps. > Regards, > Richard > >> Hi Richard, >> It is an interesting test for me too as I am planning to migrate to >> Bluestore storage and was considering repurposing the ssd disks >> that we currently use for journals. >> I was wondering if you are using the Filestore or the bluestone >> for the osds? >> Also, when you perform your testing, how good is the hit ratio >> that you have on the bcache? >> Are you using a lot of random data for your benchmarks? How >> large is your test file for each vm? >> We have been playing around with a few caching scenarios a >> few years back (enchanceio and a few more which I can't >> remember now) and we have seen a very poor hit ratio on the >> caching system. Was wondering if you see a different picture? >> Cheers > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |