Can anybody lay out a concrete use case for us employing real RCU in a Seastar OSD? When I went through the data structures, it generally seemed like message-passing about data structure changes would be a better way to go than trying to employ any kind of real RCU library (or even the exact abstractions). We might maintain local pointers to constant structures with a per-core ref count to protect deletion, but proper RCU would involve unpredictable access to out-of-core memory locations (especially if we have multiple writers!), whereas if we stick with the message-passing that Seastar expects, we get all the optimizations that come from those very careful data structures. -Greg On Fri, Jun 8, 2018 at 4:41 AM, kefu chai <tchaikov@xxxxxxxxx> wrote: > On Fri, Jun 8, 2018 at 11:08 AM Sage Weil <sweil@xxxxxxxxxx> wrote: >> >> On Fri, 8 Jun 2018, Liu, Chunmei wrote: >> > Hi Kefu, >> > >> > For RCU, I get the following facts, >> > 1. For readers, no block, and support multiple readers concurrently. >> > 2. For writers, it is blocked until all readers go out of reader-side critical session. If there are multiple concurrent writers need spin lock to synchronize all the writers. >> > >> > Are there multi concurrent writers for config or OSDMap? >> >> Not really... and for config writes are rare. I suspect we could even get >> away with something that just allocates each new value on the heap and >> updates a pointer, and then never bothers to free the old values. (Or >> maybe frees them after a ridiculous amount of time has passed.) > > yeah, RCU supports the concurrency between a single writer and > multiple readers. in the case of config options, i think the writer(s) > would be the threads/reactors serving the MCommand sent from osd > clients. so in theory, there could be multiple writers. but as Sage > pointed out, writes are rare. i think we could use spin lock to > implement the exclusive lock. > >> >> For the OSDMap cache, we have a constant stream of new maps coming in and >> old maps getting pruned, so it's a bit trickier. > > yeah, both MonClient and peer OSDs update the OSD with new maps, and > OSD keep trimming unused maps from the cache. > >> >> > Do you think it is acceptable in seastar version since need spin lock >> > for multiple concurrent writers? > > i think it's fine if the write is relatively rare and what the spin > lock protects are very fast operations, like flipping a flag or > setting a pointer, etc. > >> > >> > Thanks! >> > -Chunmei >> > -- >> > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> > the body of a message to majordomo@xxxxxxxxxxxxxxx >> > More majordomo info at http://vger.kernel.org/majordomo-info.html >> > >> > > > > > -- > Regards > Kefu Chai > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
