On 1/22/19 2:39 PM, Joel Fernandes wrote:
[snip]
On Sun, Jan 20, 2019 at 06:49:56PM -0800, hpa@xxxxxxxxx wrote:
[snip]
My point is that if we're going to actually solve a problem, we need to make it so that the distro won't just disable it anyway, and it ought to be something scalable; otherwise nothing is gained.
Agreed. Although there are many ways distros can misconfigure a kernel and
shoot themselves in the foot.
I am *not* disagreeing with the problem statement!
Ok, that's great to know. Thanks for all the discussions!
Now, /proc isn't something that will autoload modules. A filesystem *will*, although you need to be able to mount it; furthermore, it makes it trivially to extend it (and the firmware interface provides an . easy way to feed the data to such a filesystem without having to muck with anything magic.)
My thought is for the tools needing the module to try to load them, if they
need the headers. And then unload the module once they are done with it.
Heck, we could even make it a squashfs image that can just be mounted.
That would add another dependency on a kernel configuration though.
So, first of all, where does Android keep its modules, and what is actually included? Is /sbin/modprobe used to load the modules, as is normal? We might even be able to address this with some fairly trivial enhancements to modprobe; specifically to search in the module paths for something that isn't a module per se.
The best scenario would be if we could simply have the tools find the location equivalent of /lib/modules/$version/source...
Sandeep answered this in the other thread, modprobe is there in Android.
About storing sources on the filesystem, I already went down this route (this
was my first effort) and there is quite some resistance to ship sources on
the filesystem of an Android device due to Licensing reasons. A kernel module
though is something that's a binary artifact and is not something
"distributed" by Google.
The other issue is with enforcing different OEM vendors that they should
store kernel-header sources on their Android products. That's much harder to
enforce, however we can easily enforce kernel config options be enabled,
since we already enforce a kernel configuration options that are needed for
Android to work correctly:
https://source.android.com/devices/architecture/kernel/config. So this config
option can just be another one.
By the way, we can easily write a script to just extract the .ko directly -
if the whole "load it as a module" thing bothers you. The kheaders.ko can
just be thought of as a tarball. There's already a script to extract
/proc/config.gz in the same/similar way: scripts/extract-ikconfig
If I may add a few more thoughts here ... in no specific order:
I've been helping customers put Android in all sorts of weird devices
over the past decade, handsets included, and have had a front-row seat
to how this ecosystem works/has-evolved (Google, SoC vendors,
manufacturers, etc.) One thing that stands out is that while the outside
perception might be that Google has a stronghold on what "partners" do,
the reality is that almost any rule that can be broken will be broken by
someone in the ecosystem. And yet, still, Google somehow has to manage
and provide app developers and, ultimately, consumers with a coherent
experience, while still herding everyone to more-or-less go in the same
direction ...
From that point of view, if something comes from or is rooted in
mainline, instead of being mandated, it's usually easier to find it
across the board. A perfect example of this is ftrace. The fact that
it's in mainline has enabled google to: a) instrument their entire stack
to log events to it (see
http://www.opersys.com/downloads/cc-slides/android-debug/slides-main-181012.html#/82
and
http://www.opersys.com/downloads/cc-slides/android-debug/slides-main-181012.html#/83),
and b) provide app-developer-facing tools (see
https://developer.android.com/studio/command-line/systrace). Since this
tracing functionality is now integrated into Android Studio (look for
"System Trace" here:
https://developer.android.com/studio/profile/cpu-profiler), it's very
much "standard android" and additional proof, if any was needed, that
tracing is useful to everyone.
A few years back I was asked by a customer to put together some class
material for internal Android debugging and performance analysis
(commercial disclaimer applies, but slides/exercises are under
"courseware":
http://www.opersys.com/training/android-debug-and-performance). ftrace
was very much in those early versions and it was great to show people
that they could use it "out of the box". Recently I wanted to update
this class material to cover eBPF and its applicability in Android. Holy
cow. That turned out to be less obvious than necessary and somewhat
peculiar to pull off. In this specific case, Joel tried a few things
(see
http://www.opersys.com/downloads/cc-slides/android-debug/slides-main-181012.html#/111)
before eventually settling on loading a Debian chroot jail into a live
Android (https://github.com/joelagnel/adeb) ... all of which require a
proper set of kernel headers to properly function. Don't get me wrong,
Joel's Androdeb makes this easy, but it's still outside the standard
Android MO.
In short, let's just say that, contrary to ftrace, I don't see the path
for eBPF to being part of the standard toolset used by app developers
any time soon. The recently-released bpftrace might help in that regard,
but the kernel headers aren't too far in that regard either.
Personally I advocated a more aggressive approach with Joel in private:
just put the darn headers straight into the kernel image, it's the
*only* artifact we're sure will follow the Android device whatever
happens to it (like built-in ftrace). To that end, I even had some crazy
ideas on how to compress the headers even further than with std
compression algorithms -- here's a snippet from an email I sent Joel
some time back detailing such a hack:
Since C headers have fairly constrained semantics and since the types of semantics generally used to name structs, etc. in the Linux kernel are well established, we can likely devise a very customized compression algorithm for the purpose.
We know, for example, that there are quite a few ascii characters that can't be found in a normal header. So we can play some games. Say, replace all the "#" dictionary with 2 bytes, the first being a magic (illegal) ascii character and the 2nd with a constant. As such, we could replace all #foo entries with 2 bytes:
#define goes from 7 bytes to 2
#include from 8 to 2
#ifdef from 6 to 2
etc.
The same goes for several keywords of the C language and words very often found in Linux kernel headers:
struct from 6 to 2
linux from 5 to 2
CONFIG from 6 to 2
"unsigned long" from 13 to 2
etc.
And if we run out of space with one magic character (256), we can start using a next one.
We'd have a script that runs at build time that does this replacement, and we could even package the counter-script as part of the kernel image the to convert the "cooked" headers back to their original format; ex: Steven has a README in the ftrace directory.
Such a replacement strategy combined with stripping comments and gzipping the result may provide some very interesting compression results.
Whether such craziness makes sense or is adopted or not isn't mine to
chart, but I certainly can't see eBPF reaching the same mass deployment
ftrace has within the Android ecosystem until there's a way to use it
without having to chase kernel headers independently of kernel images.
There are "too many clicks" involved and someone somewhere will drop the
ball if it's not glued to the kernel in some way shape or form. Any
solution that solves this is one I'd love to hear about.
My $0.02
--
Karim Yaghmour
CEO - Opersys inc. / www.opersys.com
http://twitter.com/karimyaghmour
