On Mon, Apr 8, 2019 at 9:29 AM Olof Johansson <olof@xxxxxxxxx> wrote: > > Hi, > > On Wed, Mar 20, 2019 at 9:31 AM Joel Fernandes (Google) > <joel@xxxxxxxxxxxxxxxxx> wrote: > > > > Introduce in-kernel headers and other artifacts which are made available > > as an archive through proc (/proc/kheaders.tar.xz file). This archive makes > > it possible to build kernel modules, run eBPF programs, and other > > tracing programs that need to extend the kernel for tracing purposes > > without any dependency on the file system having headers and build > > artifacts. > > > > On Android and embedded systems, it is common to switch kernels but not > > have kernel headers available on the file system. Further once a > > different kernel is booted, any headers stored on the file system will > > no longer be useful. By storing the headers as a compressed archive > > within the kernel, we can avoid these issues that have been a hindrance > > for a long time. > > > > The best way to use this feature is by building it in. Several users > > have a need for this, when they switch debug kernels, they donot want to > > update the filesystem or worry about it where to store the headers on > > it. However, the feature is also buildable as a module in case the user > > desires it not being part of the kernel image. This makes it possible to > > load and unload the headers from memory on demand. A tracing program, or > > a kernel module builder can load the module, do its operations, and then > > unload the module to save kernel memory. The total memory needed is 3.8MB. > > > > By having the archive available at a fixed location independent of > > filesystem dependencies and conventions, all debugging tools can > > directly refer to the fixed location for the archive, without concerning > > with where the headers on a typical filesystem which significantly > > simplifies tooling that needs kernel headers. > > > > The code to read the headers is based on /proc/config.gz code and uses > > the same technique to embed the headers. > > > > To build a module, the below steps have been tested on an x86 machine: > > modprobe kheaders > > rm -rf $HOME/headers > > mkdir -p $HOME/headers > > tar -xvf /proc/kheaders.tar.xz -C $HOME/headers >/dev/null > > cd my-kernel-module > > make -C $HOME/headers M=$(pwd) modules > > rmmod kheaders > > > > Additional notes: > > (1) external modules must be built on the same arch as the host that > > built vmlinux. This can be done either in a qemu emulated chroot on the > > target, or natively. This is due to host arch dependency of kernel > > scripts. > > > > (2) > > If module building is used, since Module.symvers is not available in the > > archive due to a cyclic dependency with building of the archive into the > > kernel or module binaries, the modules built using the archive will not > > contain symbol versioning (modversion). This is usually not an issue > > since the idea of this patch is to build a kernel module on the fly and > > load it into the same kernel. An appropriate warning is already printed > > by the kernel to alert the user of modules not having modversions when > > built using the archive. For building with modversions, the user can use > > traditional header packages. For our tracing usecases, we build modules > > on the fly with this so it is not a concern. > > > > (3) I have left IKHD_ST and IKHD_ED markers as is to facilitate > > future patches that would extract the headers from a kernel or module > > image. > > > > (v4 was Tested-by the following folks, > > v5 only has minor changes and has passed my testing). > > Tested-by: qais.yousef@xxxxxxx > > Tested-by: dietmar.eggemann@xxxxxxx > > Tested-by: linux@xxxxxxxxxxxxxxxx > > Signed-off-by: Joel Fernandes (Google) <joel@xxxxxxxxxxxxxxxxx> > > Sorry to be late at the party with this kind of feedback, but I find > the whole ".tar.gz in procfs" to be an awkward solution, especially if > there's expected to be userspace tooling that depends on this > long-term. > [snip] The approaches you proposed were explored in detail on this thread and other related threads. The tarball in proc approach is a simple, pragmatic approach that allows makes a lot of scenarios Just Work where they didn't before. Approaches like a new filesystem, a mountable block device, a custom debuginfo format, and so on add complexity without providing concrete gains in functionality. We'd like to get this work into the tree sooner rather than later.
