On 30/10/2015, Jason Dillaman wrote: > In terms of overhead for the interface, is your concern due to the > potential for more heap allocations for storing tasks to complete > the future? If that's the case, Adam's std::function replacement [1], > which allocates a configurable amount of storage on the stack (as opposed > to going to the heap for functions larger than 16 bytes as in the GCC > implementation [2]), might be a good compromise. > > I had some time to deep-dive through the seastar library this week and > was really impressed by its design. I know we (semi-)recently needed to > introduce a workaround change to librbd which now results in a context > switch for each AIO op -- all to ensure that we never block the QEMU IO > thread. Long term, I would love if we could take some of these reactive > programming concepts to avoid all potential blocking on the librbd / > librados IO path (with a side benefit of perhaps eliminating the potential > for thousands of network IO threads by instead sharding the work). > > [1] http://comments.gmane.org/gmane.comp.file-systems.ceph.devel/27100 > [2] http://www.drdobbs.com/cpp/efficient-use-of-lambda-expressions-and/232500059?pgno=2 We have been working on a short run prototype/research project to see about using Seastar itself as a reimplementation of the OSD top-half that can support the semantics that Ceph's placement group system currently relies upon as well as different guarantees alongside them. This is part of our effort to repackage and upstream the Cohort low-latency OSD work in a way that will integrate better with the existing Ceph requirements. (Eric is also hoping to use this as a model for implementing and evaluating DM-CLOCK.) We did find that replacing Context* with the static-reservation function implementation greatly improved the performance of Objecter (which is what I initially designed it for), but there was still a lot of lock contention. While Seastar itself as it exists is not suitable for client applications (since it wants to take over your whole program and assumes it owns cores in their entirety) a refactoring or inspiration from Seastar that included futures, the CPU-affine allocator, their mostly lock-free approach, and their userspace network support would likely give even better gains in Objecter performance. We are not (yet) working on Objecter, but we would love to collaborate with anyone wanting to do this work. -- Adam C. Emerson, Senior Software Engineer Red Hat Storage, Ann Arbor, Michigan, United States -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
