On Wed, Nov 4, 2015 at 7:07 AM, Gregory Farnum <gfarnum@xxxxxxxxxx> wrote: > The problem with this approach is that the encoded versions need to be > platform-independent — they are shared over the wire and written to > disks that might get transplanted to different machines. Apart from > padding bytes, we also need to worry about endianness of the machine, > etc. *And* we often mutate structures across versions in order to add > new abilities, relying on the encode-decode process to deal with any > changes to the system. How could we deal with that if just dumping the > raw memory? > > Now, maybe we could make these changes on some carefully-selected > structs, I'm not sure. But we'd need a way to pick them out, guarantee > that we aren't breaking interoperability concerns, etc; and it would > need to be something we can maintain as a group going forward. I'm not > sure how to satisfy those constraints without burning a little extra > CPU. :/ > -Greg So it turns out we've actually had issues with this. Sage merged (wrote?) some little-endian-only optimizations to the cephx code that broke big-endian systems by doing a direct memcpy. Apparently our tests don't find these issues, which makes me even more nervous about taking that sort of optimization into the tree. :( -Greg On Sun, Nov 8, 2015 at 6:28 AM, Sage Weil <sage@xxxxxxxxxxxx> wrote: > On Sat, 7 Nov 2015, Haomai Wang wrote: >> Hi sage, >> >> Could we know about your progress to refactor MSubOP and hobject_t, >> pg_stat_t decode problem? >> >> We could work on this based on your work if any. > > See Piotr's last email on this thead... it has Josh's patch attached. > > sage > > >> >> >> On Thu, Nov 5, 2015 at 1:29 AM, Haomai Wang <haomai@xxxxxxxx> wrote: >> > On Thu, Nov 5, 2015 at 1:19 AM, Piotr.Dalek@xxxxxxxxxxxxxx >> > <Piotr.Dalek@xxxxxxxxxxxxxx> wrote: >> >>> -----Original Message----- >> >>> From: ceph-devel-owner@xxxxxxxxxxxxxxx [mailto:ceph-devel- >> >>> owner@xxxxxxxxxxxxxxx] On Behalf Of ??? >> >>> Sent: Wednesday, November 04, 2015 4:34 PM >> >>> To: Gregory Farnum >> >>> Cc: ceph-devel@xxxxxxxxxxxxxxx >> >>> Subject: Re: ceph encoding optimization >> >>> >> >>> I agree with pg_stat_t (and friends) is a good first start. >> >>> The eversion_t and utime_t are also good choice to start because they are >> >>> used at many places. >> >> >> >> On Ceph Hackathon, Josh Durgin made initial steps in right direction in terms of pg_stat_t encoding and decoding optimization, with the endianness-awareness thing left out. Even in that state, performance improvements offered by this change were huge enough to make it worthwhile. I'm attaching the patch, but please note that this is prototype and based on mid-August state of code, so you might need to take that into account when applying the patch. >> > >> > Cool, it's exactly we want to see. >> > >> >> >> >> >> >> With best regards / Pozdrawiam >> >> Piotr Da?ek >> >> >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majordomo@xxxxxxxxxxxxxxx >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> >> -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
