While we've been focusing the encode/decode optimization around bluestore. I'm eager to see what these new schemes will do for things on the wire. Our old performance measurements (back in the E/F timeframe) showed that wire-oriented serialization/deserialization was a significant CPU hog too. Just something to get around to once we stabilize what we're going to do in this area. Allen Samuels SanDisk |a Western Digital brand 2880 Junction Avenue, San Jose, CA 95134 T: +1 408 801 7030| M: +1 408 780 6416 allen.samuels@xxxxxxxxxxx > -----Original Message----- > From: Mark Nelson [mailto:mnelson@xxxxxxxxxx] > Sent: Thursday, August 18, 2016 7:41 PM > To: Varada Kari <Varada.Kari@xxxxxxxxxxx>; Samuel Just > <sjust@xxxxxxxxxx>; Allen Samuels <Allen.Samuels@xxxxxxxxxxx> > Cc: ceph-devel@xxxxxxxxxxxxxxx; Weil, Sage <sweil@xxxxxxxxxx> > Subject: Re: wip-enc-dec > > Hi Varada, > > FWIW, here's my branch where I've been playing with using safe_appenders > in the existing bluestore encoding scheme: > > https://github.com/markhpc/ceph/tree/wip-bluestore-append > > Looks like it's going to be pretty advantageous to use a big appender to > encode as much of the blob as possible. It's making a pretty substantial > difference, at least with ou > > Mark > > On 08/18/2016 09:03 PM, Varada Kari wrote: > > Hi Sam, > > > > Here is some wip code on the new approach proposed by Allen. > > > > https://github.com/varadakari/ceph/commits/wip-enc-dec > > > > we are using a raw buffer which has all the encoded data and absorbed > > into bufferlist once completed. > > It pretty much has all the containers and primitive types support. > > Working on some more optimizations on > > the bluestore end to reduce the onode size along with blob map, but > > they are wrapped in BlueStore.* files. > > enc_dec.h and test_enc_dec.cc contains the logic. > > > > Haven't tired with the safe and unsafe appenders yet. > > > > Varada > > > > On Friday 19 August 2016 05:44 AM, Samuel Just wrote: > >> Here's a beginning on what that might look like: > >> https://github.com/athanatos/ceph/commits/wip-sam-enc-dec-proto > >> > >> The design mostly comes from Allen/Varada's approach, but it uses > >> unsafe_appender (I don't really feel strongly about that, Sage > >> indicated that just using a raw pointer would probably be faster). > >> The important part is that the top two commits above convert all > >> primitive types as well as vector and pair (provided that their > >> templated types support the new machinery). This means that > >> vector<int> should be able to preallocate and use unsafe_appender for > >> all existing users. I think it's worth going to some effort to make > >> it easy to convert over existing users. > >> -Sam > >> > >> On Thu, Aug 18, 2016 at 10:35 AM, Allen Samuels > >> <Allen.Samuels@xxxxxxxxxxx> wrote: > >>> No reason not to merge them. > >>> > >>> > >>> Allen Samuels > >>> SanDisk |a Western Digital brand > >>> 2880 Junction Avenue, San Jose, CA 95134 > >>> T: +1 408 801 7030| M: +1 408 780 6416 allen.samuels@xxxxxxxxxxx > >>> > >>> > >>>> -----Original Message----- > >>>> From: Samuel Just [mailto:sjust@xxxxxxxxxx] > >>>> Sent: Thursday, August 18, 2016 9:43 AM > >>>> To: Allen Samuels <Allen.Samuels@xxxxxxxxxxx>; Varada Kari > >>>> <Varada.Kari@xxxxxxxxxxx>; ceph-devel@xxxxxxxxxxxxxxx > >>>> Subject: wip-enc-dec > >>>> > >>>> Is there a reason not to put this new machinery directly into > encoding.h? The > >>>> new machinery seems to be sufficient to implement the existing > ::encode > >>>> and ::decode types and should be compatible, right? > >>>> With the containers, we can use template trickery to also forward the > new > >>>> functions when present in the mapped types. That would let us > gradually > >>>> adapt the code base. > >>>> -Sam > > > > PLEASE NOTE: The information contained in this electronic mail message is > intended only for the use of the designated recipient(s) named above. If the > reader of this message is not the intended recipient, you are hereby notified > that you have received this message in error and that any review, > dissemination, distribution, or copying of this message is strictly prohibited. If > you have received this communication in error, please notify the sender by > telephone or e-mail (as shown above) immediately and destroy any and all > copies of this message in your possession (whether hard copies or > electronically stored copies). > > -- > > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > > the body of a message to majordomo@xxxxxxxxxxxxxxx > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
