Yep! I'm *very* excited to see where all of this leads.
Mark
On 08/18/2016 09:43 PM, Allen Samuels wrote:
While we've been focusing the encode/decode optimization around bluestore. I'm eager to see what these new schemes will do for things on the wire. Our old performance measurements (back in the E/F timeframe) showed that wire-oriented serialization/deserialization was a significant CPU hog too.
Just something to get around to once we stabilize what we're going to do in this area.
Allen Samuels
SanDisk |a Western Digital brand
2880 Junction Avenue, San Jose, CA 95134
T: +1 408 801 7030| M: +1 408 780 6416
allen.samuels@xxxxxxxxxxx
-----Original Message-----
From: Mark Nelson [mailto:mnelson@xxxxxxxxxx]
Sent: Thursday, August 18, 2016 7:41 PM
To: Varada Kari <Varada.Kari@xxxxxxxxxxx>; Samuel Just
<sjust@xxxxxxxxxx>; Allen Samuels <Allen.Samuels@xxxxxxxxxxx>
Cc: ceph-devel@xxxxxxxxxxxxxxx; Weil, Sage <sweil@xxxxxxxxxx>
Subject: Re: wip-enc-dec
Hi Varada,
FWIW, here's my branch where I've been playing with using safe_appenders
in the existing bluestore encoding scheme:
https://github.com/markhpc/ceph/tree/wip-bluestore-append
Looks like it's going to be pretty advantageous to use a big appender to
encode as much of the blob as possible. It's making a pretty substantial
difference, at least with ou
Mark
On 08/18/2016 09:03 PM, Varada Kari wrote:
Hi Sam,
Here is some wip code on the new approach proposed by Allen.
https://github.com/varadakari/ceph/commits/wip-enc-dec
we are using a raw buffer which has all the encoded data and absorbed
into bufferlist once completed.
It pretty much has all the containers and primitive types support.
Working on some more optimizations on
the bluestore end to reduce the onode size along with blob map, but
they are wrapped in BlueStore.* files.
enc_dec.h and test_enc_dec.cc contains the logic.
Haven't tired with the safe and unsafe appenders yet.
Varada
On Friday 19 August 2016 05:44 AM, Samuel Just wrote:
Here's a beginning on what that might look like:
https://github.com/athanatos/ceph/commits/wip-sam-enc-dec-proto
The design mostly comes from Allen/Varada's approach, but it uses
unsafe_appender (I don't really feel strongly about that, Sage
indicated that just using a raw pointer would probably be faster).
The important part is that the top two commits above convert all
primitive types as well as vector and pair (provided that their
templated types support the new machinery). This means that
vector<int> should be able to preallocate and use unsafe_appender for
all existing users. I think it's worth going to some effort to make
it easy to convert over existing users.
-Sam
On Thu, Aug 18, 2016 at 10:35 AM, Allen Samuels
<Allen.Samuels@xxxxxxxxxxx> wrote:
No reason not to merge them.
Allen Samuels
SanDisk |a Western Digital brand
2880 Junction Avenue, San Jose, CA 95134
T: +1 408 801 7030| M: +1 408 780 6416 allen.samuels@xxxxxxxxxxx
-----Original Message-----
From: Samuel Just [mailto:sjust@xxxxxxxxxx]
Sent: Thursday, August 18, 2016 9:43 AM
To: Allen Samuels <Allen.Samuels@xxxxxxxxxxx>; Varada Kari
<Varada.Kari@xxxxxxxxxxx>; ceph-devel@xxxxxxxxxxxxxxx
Subject: wip-enc-dec
Is there a reason not to put this new machinery directly into
encoding.h? The
new machinery seems to be sufficient to implement the existing
::encode
and ::decode types and should be compatible, right?
With the containers, we can use template trickery to also forward the
new
functions when present in the mapped types. That would let us
gradually
adapt the code base.
-Sam
PLEASE NOTE: The information contained in this electronic mail message is
intended only for the use of the designated recipient(s) named above. If the
reader of this message is not the intended recipient, you are hereby notified
that you have received this message in error and that any review,
dissemination, distribution, or copying of this message is strictly prohibited. If
you have received this communication in error, please notify the sender by
telephone or e-mail (as shown above) immediately and destroy any and all
copies of this message in your possession (whether hard copies or
electronically stored copies).
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
