Hi Varada,
FWIW, here's my branch where I've been playing with using safe_appenders
in the existing bluestore encoding scheme:
https://github.com/markhpc/ceph/tree/wip-bluestore-append
Looks like it's going to be pretty advantageous to use a big appender to
encode as much of the blob as possible. It's making a pretty
substantial difference, at least with ou
Mark
On 08/18/2016 09:03 PM, Varada Kari wrote:
Hi Sam,
Here is some wip code on the new approach proposed by Allen.
https://github.com/varadakari/ceph/commits/wip-enc-dec
we are using a raw buffer which has all the encoded data and absorbed
into bufferlist once completed.
It pretty much has all the containers and primitive types support.
Working on some more optimizations on
the bluestore end to reduce the onode size along with blob map, but they
are wrapped in BlueStore.* files.
enc_dec.h and test_enc_dec.cc contains the logic.
Haven't tired with the safe and unsafe appenders yet.
Varada
On Friday 19 August 2016 05:44 AM, Samuel Just wrote:
Here's a beginning on what that might look like:
https://github.com/athanatos/ceph/commits/wip-sam-enc-dec-proto
The design mostly comes from Allen/Varada's approach, but it uses
unsafe_appender (I don't really feel strongly about that, Sage
indicated that just using a raw pointer would probably be faster).
The important part is that the top two commits above convert all
primitive types as well as vector and pair (provided that their
templated types support the new machinery). This means that
vector<int> should be able to preallocate and use unsafe_appender for
all existing users. I think it's worth going to some effort to make
it easy to convert over existing users.
-Sam
On Thu, Aug 18, 2016 at 10:35 AM, Allen Samuels
<Allen.Samuels@xxxxxxxxxxx> wrote:
No reason not to merge them.
Allen Samuels
SanDisk |a Western Digital brand
2880 Junction Avenue, San Jose, CA 95134
T: +1 408 801 7030| M: +1 408 780 6416
allen.samuels@xxxxxxxxxxx
-----Original Message-----
From: Samuel Just [mailto:sjust@xxxxxxxxxx]
Sent: Thursday, August 18, 2016 9:43 AM
To: Allen Samuels <Allen.Samuels@xxxxxxxxxxx>; Varada Kari
<Varada.Kari@xxxxxxxxxxx>; ceph-devel@xxxxxxxxxxxxxxx
Subject: wip-enc-dec
Is there a reason not to put this new machinery directly into encoding.h? The
new machinery seems to be sufficient to implement the existing ::encode
and ::decode types and should be compatible, right?
With the containers, we can use template trickery to also forward the new
functions when present in the mapped types. That would let us gradually
adapt the code base.
-Sam
PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies).
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
