On Fri, 19 Aug 2016, Allen Samuels wrote: > While we've been focusing the encode/decode optimization around > bluestore. I'm eager to see what these new schemes will do for things on > the wire. Our old performance measurements (back in the E/F timeframe) > showed that wire-oriented serialization/deserialization was a > significant CPU hog too. Yes! :) s > Just something to get around to once we stabilize what we're going to do > in this area. > > > Allen Samuels > SanDisk |a Western Digital brand > 2880 Junction Avenue, San Jose, CA 95134 > T: +1 408 801 7030| M: +1 408 780 6416 > allen.samuels@xxxxxxxxxxx > > > > -----Original Message----- > > From: Mark Nelson [mailto:mnelson@xxxxxxxxxx] > > Sent: Thursday, August 18, 2016 7:41 PM > > To: Varada Kari <Varada.Kari@xxxxxxxxxxx>; Samuel Just > > <sjust@xxxxxxxxxx>; Allen Samuels <Allen.Samuels@xxxxxxxxxxx> > > Cc: ceph-devel@xxxxxxxxxxxxxxx; Weil, Sage <sweil@xxxxxxxxxx> > > Subject: Re: wip-enc-dec > > > > Hi Varada, > > > > FWIW, here's my branch where I've been playing with using safe_appenders > > in the existing bluestore encoding scheme: > > > > https://github.com/markhpc/ceph/tree/wip-bluestore-append > > > > Looks like it's going to be pretty advantageous to use a big appender to > > encode as much of the blob as possible. It's making a pretty substantial > > difference, at least with ou > > > > Mark > > > > On 08/18/2016 09:03 PM, Varada Kari wrote: > > > Hi Sam, > > > > > > Here is some wip code on the new approach proposed by Allen. > > > > > > https://github.com/varadakari/ceph/commits/wip-enc-dec > > > > > > we are using a raw buffer which has all the encoded data and absorbed > > > into bufferlist once completed. > > > It pretty much has all the containers and primitive types support. > > > Working on some more optimizations on > > > the bluestore end to reduce the onode size along with blob map, but > > > they are wrapped in BlueStore.* files. > > > enc_dec.h and test_enc_dec.cc contains the logic. > > > > > > Haven't tired with the safe and unsafe appenders yet. > > > > > > Varada > > > > > > On Friday 19 August 2016 05:44 AM, Samuel Just wrote: > > >> Here's a beginning on what that might look like: > > >> https://github.com/athanatos/ceph/commits/wip-sam-enc-dec-proto > > >> > > >> The design mostly comes from Allen/Varada's approach, but it uses > > >> unsafe_appender (I don't really feel strongly about that, Sage > > >> indicated that just using a raw pointer would probably be faster). > > >> The important part is that the top two commits above convert all > > >> primitive types as well as vector and pair (provided that their > > >> templated types support the new machinery). This means that > > >> vector<int> should be able to preallocate and use unsafe_appender for > > >> all existing users. I think it's worth going to some effort to make > > >> it easy to convert over existing users. > > >> -Sam > > >> > > >> On Thu, Aug 18, 2016 at 10:35 AM, Allen Samuels > > >> <Allen.Samuels@xxxxxxxxxxx> wrote: > > >>> No reason not to merge them. > > >>> > > >>> > > >>> Allen Samuels > > >>> SanDisk |a Western Digital brand > > >>> 2880 Junction Avenue, San Jose, CA 95134 > > >>> T: +1 408 801 7030| M: +1 408 780 6416 allen.samuels@xxxxxxxxxxx > > >>> > > >>> > > >>>> -----Original Message----- > > >>>> From: Samuel Just [mailto:sjust@xxxxxxxxxx] > > >>>> Sent: Thursday, August 18, 2016 9:43 AM > > >>>> To: Allen Samuels <Allen.Samuels@xxxxxxxxxxx>; Varada Kari > > >>>> <Varada.Kari@xxxxxxxxxxx>; ceph-devel@xxxxxxxxxxxxxxx > > >>>> Subject: wip-enc-dec > > >>>> > > >>>> Is there a reason not to put this new machinery directly into > > encoding.h? The > > >>>> new machinery seems to be sufficient to implement the existing > > ::encode > > >>>> and ::decode types and should be compatible, right? > > >>>> With the containers, we can use template trickery to also forward the > > new > > >>>> functions when present in the mapped types. That would let us > > gradually > > >>>> adapt the code base. > > >>>> -Sam > > > > > > PLEASE NOTE: The information contained in this electronic mail message is > > intended only for the use of the designated recipient(s) named above. If the > > reader of this message is not the intended recipient, you are hereby notified > > that you have received this message in error and that any review, > > dissemination, distribution, or copying of this message is strictly prohibited. If > > you have received this communication in error, please notify the sender by > > telephone or e-mail (as shown above) immediately and destroy any and all > > copies of this message in your possession (whether hard copies or > > electronically stored copies). > > > -- > > > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > > > the body of a message to majordomo@xxxxxxxxxxxxxxx > > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > > > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
