Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




On 1/22/19 2:39 PM, Joel Fernandes wrote:
[snip]
On Sun, Jan 20, 2019 at 06:49:56PM -0800, hpa@xxxxxxxxx wrote:
[snip]
My point is that if we're going to actually solve a problem, we need to make it so that the distro won't just disable it anyway, and it ought to be something scalable; otherwise nothing is gained.

Agreed. Although there are many ways distros can misconfigure a kernel and
shoot themselves in the foot.

I am *not* disagreeing with the problem statement!

Ok, that's great to know. Thanks for all the discussions!

Now, /proc isn't something that will autoload modules. A filesystem *will*, although you need to be able to mount it; furthermore, it makes it trivially to extend it (and the firmware interface provides an . easy way to feed the data to such a filesystem without having to muck with anything magic.)

My thought is for the tools needing the module to try to load them, if they
need the headers. And then unload the module once they are done with it.

Heck, we could even make it a squashfs image that can just be mounted.

That would add another dependency on a kernel configuration though.

So, first of all, where does Android keep its modules, and what is actually included? Is /sbin/modprobe used to load the modules, as is normal? We might even be able to address this with some fairly trivial enhancements to modprobe; specifically to search in the module paths for something that isn't a module per se.
The best scenario would be if we could simply have the tools find the location equivalent of /lib/modules/$version/source...

Sandeep answered this in the other thread, modprobe is there in Android.
About storing sources on the filesystem, I already went down this route (this
was my first effort) and there is quite some resistance to ship sources on
the filesystem of an Android device due to Licensing reasons. A kernel module
though is something that's a binary artifact and is not something
"distributed" by Google.

The other issue is with enforcing different OEM vendors that they should
store kernel-header sources on their Android products. That's much harder to
enforce, however we can easily enforce kernel config options be enabled,
since we already enforce a kernel configuration options that are needed for
Android to work correctly:
https://source.android.com/devices/architecture/kernel/config. So this config
option can just be another one.

By the way, we can easily write a script to just extract the .ko directly -
if the whole "load it as a module" thing bothers you. The kheaders.ko can
just be thought of as a tarball. There's already a script to extract
/proc/config.gz in the same/similar way: scripts/extract-ikconfig

If I may add a few more thoughts here ... in no specific order:

I've been helping customers put Android in all sorts of weird devices over the past decade, handsets included, and have had a front-row seat to how this ecosystem works/has-evolved (Google, SoC vendors, manufacturers, etc.) One thing that stands out is that while the outside perception might be that Google has a stronghold on what "partners" do, the reality is that almost any rule that can be broken will be broken by someone in the ecosystem. And yet, still, Google somehow has to manage and provide app developers and, ultimately, consumers with a coherent experience, while still herding everyone to more-or-less go in the same direction ...

From that point of view, if something comes from or is rooted in mainline, instead of being mandated, it's usually easier to find it across the board. A perfect example of this is ftrace. The fact that it's in mainline has enabled google to: a) instrument their entire stack to log events to it (see http://www.opersys.com/downloads/cc-slides/android-debug/slides-main-181012.html#/82 and http://www.opersys.com/downloads/cc-slides/android-debug/slides-main-181012.html#/83), and b) provide app-developer-facing tools (see https://developer.android.com/studio/command-line/systrace). Since this tracing functionality is now integrated into Android Studio (look for "System Trace" here: https://developer.android.com/studio/profile/cpu-profiler), it's very much "standard android" and additional proof, if any was needed, that tracing is useful to everyone.

A few years back I was asked by a customer to put together some class material for internal Android debugging and performance analysis (commercial disclaimer applies, but slides/exercises are under "courseware": http://www.opersys.com/training/android-debug-and-performance). ftrace was very much in those early versions and it was great to show people that they could use it "out of the box". Recently I wanted to update this class material to cover eBPF and its applicability in Android. Holy cow. That turned out to be less obvious than necessary and somewhat peculiar to pull off. In this specific case, Joel tried a few things (see http://www.opersys.com/downloads/cc-slides/android-debug/slides-main-181012.html#/111) before eventually settling on loading a Debian chroot jail into a live Android (https://github.com/joelagnel/adeb) ... all of which require a proper set of kernel headers to properly function. Don't get me wrong, Joel's Androdeb makes this easy, but it's still outside the standard Android MO.

In short, let's just say that, contrary to ftrace, I don't see the path for eBPF to being part of the standard toolset used by app developers any time soon. The recently-released bpftrace might help in that regard, but the kernel headers aren't too far in that regard either.

Personally I advocated a more aggressive approach with Joel in private: just put the darn headers straight into the kernel image, it's the *only* artifact we're sure will follow the Android device whatever happens to it (like built-in ftrace). To that end, I even had some crazy ideas on how to compress the headers even further than with std compression algorithms -- here's a snippet from an email I sent Joel some time back detailing such a hack:
Since C headers have fairly constrained semantics and since the types of semantics generally used to name structs, etc. in the Linux kernel are well established, we can likely devise a very customized compression algorithm for the purpose.

We know, for example, that there are quite a few ascii characters that can't be found in a normal header. So we can play some games. Say, replace all the "#" dictionary with 2 bytes, the first being a magic (illegal) ascii character and the 2nd with a constant. As such, we could replace all #foo entries with 2 bytes:
#define goes from 7 bytes to 2
#include from 8 to 2
#ifdef from 6 to 2
etc.

The same goes for several keywords of the C language and words very often found in Linux kernel headers:
struct from 6 to 2
linux from 5 to 2
CONFIG from 6 to 2
"unsigned long" from 13 to 2
etc.

And if we run out of space with one magic character (256), we can start using a next one.

We'd have a script that runs at build time that does this replacement, and we could even package the counter-script as part of the kernel image the to convert the "cooked" headers back to their original format; ex: Steven has a README in the ftrace directory.

Such a replacement strategy combined with stripping comments and gzipping the result may provide some very interesting compression results.

Whether such craziness makes sense or is adopted or not isn't mine to chart, but I certainly can't see eBPF reaching the same mass deployment ftrace has within the Android ecosystem until there's a way to use it without having to chase kernel headers independently of kernel images. There are "too many clicks" involved and someone somewhere will drop the ball if it's not glued to the kernel in some way shape or form. Any solution that solves this is one I'd love to hear about.

My $0.02

--
Karim Yaghmour
CEO - Opersys inc. / www.opersys.com
http://twitter.com/karimyaghmour



[Index of Archives]     [Kernel Newbies]     [Security]     [Netfilter]     [Bugtraq]     [Linux FS]     [Yosemite Forum]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Samba]     [Video 4 Linux]     [Device Mapper]     [Linux Resources]

  Powered by Linux