100%, because of it's a small rbd image. all metadata should be cached. On Thu, May 19, 2016 at 10:39 AM, Jianjian Huo <jianjian.huo@xxxxxxxxxxx> wrote: > > On Wed, May 18, 2016 at 6:35 PM, Haomai Wang <haomai@xxxxxxxx> wrote: >> On Thu, May 19, 2016 at 7:50 AM, Jianjian Huo <jianjian.huo@xxxxxxxxxxx> wrote: >>> >>> On Wed, May 18, 2016 at 10:19 AM, Sage Weil <sage@xxxxxxxxxxxx> wrote: >>>> >>>> Hi Rajath! >>>> >>>> Great to hear you're interested in working with us outside of GSoC! >>>> >>>> On Wed, 18 May 2016, Haomai Wang wrote: >>>> > Hi Rajath, >>>> > >>>> > We are glad to see your passion, from my view, sage is planning to >>>> > implement a userspace cache in bluestore itself. Like >>>> > (https://github.com/ceph/ceph/commit/b9ac31afe5a162176f019baa25348014a77f6ab8#commitcomment-17488250). >>>> > >>>> > I guess the cache won't be a generic cache interface. Instead it will >>>> > be bound to specified needed object. So sage may give a brief? >>>> >>>> Part of the reason why this project wasn't at the top of our list (we got >>>> fewer slots than we had projects) was because the BlueStore code is in >>>> flux and moving quite quickly. For the BlueStore side, we are building a >>>> simple buffer cache that is tied to an Onode (in-memory per-object >>>> metadata structure) and integrated tightly with the read and write IO >>>> paths. This will eliminate our use of the block device buffer cache for >>>> user/object data. >>>> >>>> The other half of the picture, though, is the BlueFS layer that is >>>> consumed by rocksdb: it also needs caching in order for rocksdb to perform >>>> at all. My hope is that the code we write for the use data can be re-used >>>> here as well, but it is still evolving. >>> >>> When Bluestore moves away from kernel cache to its own buffer cache, RocksDB can use its own buffer cache as well. >>> RocksDB has this size configurable block cache to cache uncompressed data blocks, it can serve as buffer cache, >>> since Bluestore don't compress meta data in RocksDB. >> >> Actually this is not behaviored as expected. From my last nvmedevice >> benchmark, lots of read still go down device instead of caching by >> rocksdb when I set a very large block cache. I guess there exists some >> gaps between our usages and rocksdb implementation > > What kind of workload did you use for that benchmarking, 100% read? > >> >>> >>> Jianjian >>>> >>>> The main missing piece I'd say is a way to string Buffer objects together >>>> in a global(ish) LRU (or set of LRUs, or whatever we need for the caching >>>> policy that makes sense) so that trimming can be done safely and >>>> efficiently. Right now the code is lock-free because each Onode is only >>>> touched under the collection rwlock, but in order to do trimming we need >>>> to be able to reap cold buffers from a global context. >>>> >>>> Anyway, there is no clear or ready answer here yet, but we are ready to >>>> discuss design/approach here on the list, and welcome your input (and >>>> potentially, contributions to development!). >>>> >>>> sage >>>> >>>> >>>> > >>>> > On Wed, May 18, 2016 at 9:32 PM, Haomai Wang <haomai@xxxxxxxx> wrote: >>>> > > >>>> > > >>>> > > On Wed, May 18, 2016 at 2:44 PM, Rajath Shashidhara >>>> > > <rajath.shashidhara@xxxxxxxxx> wrote: >>>> > >> >>>> > >> Hello, >>>> > >> >>>> > >> I was a GSoC'16 applicant for the project "Implementing Cache layer on >>>> > >> top of NVME Driver". Unfortunately, I was not selected for the >>>> > >> internship. >>>> > > >>>> > > >>>> > > Hi Rajath, >>>> > > >>>> > > We are glad to see your passion, from my view, sage is planning to implement >>>> > > a userspace cache in bluestore itself. Like >>>> > > (https://github.com/ceph/ceph/commit/b9ac31afe5a162176f019baa25348014a77f6ab8#commitcomment-17488250). >>>> > > >>>> > > I guess the cache won't be a generic cache interface. Instead it will be >>>> > > bound to specified needed object. So sage may give a brief? >>>> > > >>>> > >> >>>> > >> >>>> > >> However, I would be interested in working on the project as an >>>> > >> independent contributor to Ceph. >>>> > >> I am expecting to receive the necessary support from Ceph developer >>>> > >> community. >>>> > >> >>>> > >> In case I missed out any important details in my project proposal or I >>>> > >> have the wrong understanding of the project, please help me figure out >>>> > >> the details. >>>> > >> >>>> > >> Looking forward to contribute! >>>> > >> >>>> > >> Thank you, >>>> > >> Rajath Shashidhara >>>> > > >>>> > > >>>> > -- >>>> > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>>> > the body of a message to majordomo@xxxxxxxxxxxxxxx >>>> > More majordomo info at http://vger.kernel.org/majordomo-info.html >>>> > >>>> > >>>> -- >>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>>> the body of a message to majordomo@xxxxxxxxxxxxxxx >>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majordomo@xxxxxxxxxxxxxxx >> More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
