On Wed, Jan 23, 2019 at 02:37:47PM -0800, Daniel Colascione wrote: > On Wed, Jan 23, 2019 at 1:29 PM Karim Yaghmour > <karim.yaghmour@xxxxxxxxxxx> wrote: [...] > > Personally I advocated a more aggressive approach with Joel in private: > > just put the darn headers straight into the kernel image, it's the > > *only* artifact we're sure will follow the Android device whatever > > happens to it (like built-in ftrace). > > I was thinking along similar lines. Ordinarily, we make loadable > kernel modules. What we kind of want here is a non-loadable kernel > module --- or a non-loadable section in the kernel image proper. I'm > not familiar with early-stage kernel loader operation: I know it's > possible to crease discardable sections in the kernel image, but can > we create sections that are never slurped into memory in the first > place? If not, maybe loading and immediately discarding the header > section is good enough. I am happy to see if I can shrink it down further. Especially using xz and stripping all comments period. I am optimistic this can be brought down further to a point where it would make sense to everyone to build it into the kernel. Lets see. Last time I stripped comments, it went down by ~40%. What I haven't tried is doing this *with* xz compression. I am also open to brainstorming what else can be stripped. OTOH the reason I didn't focus much on size is: modules are pretty much universal and I'm confident of wide spread use of this feature for Android-based products and not needing of "chasing headers" if we modularize it, since Android's project treble has modularized things and modules are now default enabled. Putting headers into a module lets us enjoy the ride there. I am quite hosed for the next week or so to work on this, but I should be able to get back to it after. cheers, - Joel > > > To that end, I even had some crazy > > ideas on how to compress the headers even further than with std > > compression algorithms -- here's a snippet from an email I sent Joel > > some time back detailing such a hack: > > > Since C headers have fairly constrained semantics and since the types of semantics generally used to name structs, etc. in the Linux kernel are well established, we can likely devise a very customized compression algorithm for the purpose. > > Would such a thing really do better than LZMA? LZMA already has very > clever techniques for eliminating long-range redundancies in > compressible text, including redundancies at the sub-byte level. I can > certainly understand the benefit of stripping comments, since removing > comments really does decrease the total amount of information the > compressor has to preserve, but I'm not sure how much the encoding > scheme you propose below would help, since it reminds me of the > encoding scheme that LZMA would discover automatically. > > > Whether such craziness makes sense or is adopted or not isn't mine to > > chart, but I certainly can't see eBPF reaching the same mass deployment > > ftrace has within the Android ecosystem until there's a way to use it > > without having to chase kernel headers independently of kernel images. > > There are "too many clicks" involved and someone somewhere will drop the > > ball if it's not glued to the kernel in some way shape or form. Any > > solution that solves this is one I'd love to hear about. > > I agree. There definitely needs to be a "just collect a damn trace" > button that works on any device, and for this button to work and > incorporate eBPF, the system needs to be able to describe itself.
