Hi Greg, How to use message-passing? each core maintain a local replication copy of data structure and use message-passing to inform other cores update its own local copy. Or only one core can access data structure, the other cores should get shared data structure through this core? Thanks! -Chunmei -----Original Message----- From: ceph-devel-owner@xxxxxxxxxxxxxxx [mailto:ceph-devel-owner@xxxxxxxxxxxxxxx] On Behalf Of Matt Benjamin Sent: Friday, June 08, 2018 11:40 AM To: Gregory Farnum <gfarnum@xxxxxxxxxx> Cc: Kefu Chai <kchai@xxxxxxxxxx>; Sage Weil <sweil@xxxxxxxxxx>; Liu, Chunmei <chunmei.liu@xxxxxxxxx>; The Esoteric Order of the Squid Cybernetic <ceph-devel@xxxxxxxxxxxxxxx> Subject: Re: using RCU to replace Locker in config for seastar version That's what I would have thought, yes. I thought the discussion was about RCU in the pthreaded codebase. Dan Lambright prototyped that for one of the maps with liburcu, a good while ago. Matt On Fri, Jun 8, 2018 at 2:35 PM, Gregory Farnum <gfarnum@xxxxxxxxxx> wrote: > Can anybody lay out a concrete use case for us employing real RCU in a > Seastar OSD? > When I went through the data structures, it generally seemed like > message-passing about data structure changes would be a better way to > go than trying to employ any kind of real RCU library (or even the > exact abstractions). We might maintain local pointers to constant > structures with a per-core ref count to protect deletion, but proper > RCU would involve unpredictable access to out-of-core memory locations > (especially if we have multiple writers!), whereas if we stick with > the message-passing that Seastar expects, we get all the optimizations > that come from those very careful data structures. > -Greg > > On Fri, Jun 8, 2018 at 4:41 AM, kefu chai <tchaikov@xxxxxxxxx> wrote: >> On Fri, Jun 8, 2018 at 11:08 AM Sage Weil <sweil@xxxxxxxxxx> wrote: >>> >>> On Fri, 8 Jun 2018, Liu, Chunmei wrote: >>> > Hi Kefu, >>> > >>> > For RCU, I get the following facts, 1. For readers, no block, >>> > and support multiple readers concurrently. >>> > 2. For writers, it is blocked until all readers go out of reader-side critical session. If there are multiple concurrent writers need spin lock to synchronize all the writers. >>> > >>> > Are there multi concurrent writers for config or OSDMap? >>> >>> Not really... and for config writes are rare. I suspect we could >>> even get away with something that just allocates each new value on >>> the heap and updates a pointer, and then never bothers to free the >>> old values. (Or maybe frees them after a ridiculous amount of time >>> has passed.) >> >> yeah, RCU supports the concurrency between a single writer and >> multiple readers. in the case of config options, i think the >> writer(s) would be the threads/reactors serving the MCommand sent >> from osd clients. so in theory, there could be multiple writers. but >> as Sage pointed out, writes are rare. i think we could use spin lock >> to implement the exclusive lock. >> >>> >>> For the OSDMap cache, we have a constant stream of new maps coming >>> in and old maps getting pruned, so it's a bit trickier. >> >> yeah, both MonClient and peer OSDs update the OSD with new maps, and >> OSD keep trimming unused maps from the cache. >> >>> >>> > Do you think it is acceptable in seastar version since need spin >>> > lock for multiple concurrent writers? >> >> i think it's fine if the write is relatively rare and what the spin >> lock protects are very fast operations, like flipping a flag or >> setting a pointer, etc. >> >>> > >>> > Thanks! >>> > -Chunmei >>> > -- >>> > To unsubscribe from this list: send the line "unsubscribe >>> > ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx >>> > More majordomo info at http://vger.kernel.org/majordomo-info.html >>> > >>> > >> >> >> >> -- >> Regards >> Kefu Chai >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" >> in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo >> info at http://vger.kernel.org/majordomo-info.html > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" > in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo > info at http://vger.kernel.org/majordomo-info.html -- Matt Benjamin Red Hat, Inc. 315 West Huron Street, Suite 140A Ann Arbor, Michigan 48103 http://www.redhat.com/en/technologies/storage tel. 734-821-5101 fax. 734-769-8938 cel. 734-216-5309 -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html ��.n��������+%������w��{.n����z��u���ܨ}���Ơz�j:+v�����w����ޙ��&�)ߡ�a����z�ޗ���ݢj��w�f
