On January 25, 2019 11:15:52 AM PST, Daniel Colascione <dancol@xxxxxxxxxx> wrote: >On Fri, Jan 25, 2019 at 11:01 AM <hpa@xxxxxxxxx> wrote: >> >> On January 24, 2019 12:59:29 PM PST, Joel Fernandes ><joel@xxxxxxxxxxxxxxxxx> wrote: >> >On Thu, Jan 24, 2019 at 07:57:26PM +0100, Karim Yaghmour wrote: >> >> >> >> On 1/23/19 11:37 PM, Daniel Colascione wrote: >> >[..] >> >> > > Personally I advocated a more aggressive approach with Joel in >> >private: >> >> > > just put the darn headers straight into the kernel image, it's >> >the >> >> > > *only* artifact we're sure will follow the Android device >> >whatever >> >> > > happens to it (like built-in ftrace). >> >> > >> >> > I was thinking along similar lines. Ordinarily, we make loadable >> >> > kernel modules. What we kind of want here is a non-loadable >kernel >> >> > module --- or a non-loadable section in the kernel image proper. >> >I'm >> >> > not familiar with early-stage kernel loader operation: I know >it's >> >> > possible to crease discardable sections in the kernel image, but >> >can >> >> > we create sections that are never slurped into memory in the >first >> >> > place? If not, maybe loading and immediately discarding the >header >> >> > section is good enough. >> >> >> >> Interesting. Maybe just append it to the image but have it not >loaded >> >and >> >> have a kernel parameter than enables a "/proc/kheaders" driver to >> >know where >> >> the fetch the appended headers from storage at runtime. There >would >> >be no >> >> RAM loading whatsoever of the headers, just some sort of >> >> "kheaders=/dev/foobar:offset:size" parameter. If you turn the >option >> >on, you >> >> get a fatter kernel image size to store on permanent storage, but >no >> >impact >> >> on what's loaded at boot time. >> > >> >Embedding anything into the kernel image does impact boot time >though >> >because >> >it increase the time spent by bootloader. A module OTOH would not >have >> >such >> >overhead. >> > >> >Also a kernel can be booted in any number of ways other than mass >> >storage so >> >it is not a generic Linux-wide solution to have a kheaders= option >like >> >that. >> >If the option is forgotten, then the running system can't use the >> >feature. >> >The other issue is it requires a kernel command line option / >> >bootloader >> >changes for that which adds more configuration burden, which not be >> >needed >> >with a module. >> > >> >> > Would such a thing really do better than LZMA? LZMA already has >> >very >> >> > clever techniques for eliminating long-range redundancies in >> >> > compressible text, including redundancies at the sub-byte level. >I >> >can >> >> > certainly understand the benefit of stripping comments, since >> >removing >> >> > comments really does decrease the total amount of information >the >> >> > compressor has to preserve, but I'm not sure how much the >encoding >> >> > scheme you propose below would help, since it reminds me of the >> >> > encoding scheme that LZMA would discover automatically. >> >> >> >> I'm no compression algorithm expert. If you say LZMA would do the >> >> same/better than what I suggested then I have no reason to contest >> >that. My >> >> goal is to see the headers as part of the kernel image that's >> >distributed on >> >> devices so that they don't have to be chased around. I'm just >trying >> >to make >> >> it as palatable as possible. >> > >> >I believe LZMA is really good at that sort of thing too. >> > >> >Also at 3.3MB of module size, I think we are really good size-wise. >But >> >Dan >> >is helping look at possibly reducing further if he gets time. Many >> >modules in >> >my experience are much bigger. amdgpu.ko on my Linux machine is >6.1MB. >> > >> >I really think making it a module is the best way to make sure this >is >> >bundled with the kernel on the widest number of Android and other >Linux >> >systems, without incurring boot time overhead, or any other command >> >line >> >configuration burden. >> > >> >I spoke to so many people at LPC personally with other kernel >> >contributors, >> >and many folks told me one word - MODULE :D. Even though I >hesitated >> >at >> >first, now it seems the right solution. >> > >> >If no one seriously objects, I'll clean this up and post a v2 and >with >> >the >> >RFC tag taken off. Thank you! >> > >> > - Joel >> >> So let me throw in a different notion. >> >> A kernel module really is nothing other than a kernel build system >artifact stored in the filesystem. >> >> I really don't at any reason whatsoever why this is direct from just >producing an archive and putting it in the module directory, except >that the latter is far simpler. >> >> I see literally *no* problem, social or technical, you are solvin by >actually making it a kernel ELF object. > >Joel does have a point. Suppose we're on Android and we're testing a >random local kernel we've built. We can command the device to boot >into the bootloader, then send our locally-built kernel to the device >with "fastboot boot mykernelz". Having booted the device this way, >there's no on-disk artifact corresponding to mykernelz: we just sent >the boot kernel directly to the device's memory. Now, suppose I want >to attach DCTV or some other fancy ftrace-based analysis tool to the >device in order to study how mykernelz acts in some scenario I care >about. With Joel's approach, DCTV would be able to grab the kernel >headers from the running kernel, compile whatever kprobe or BPF >incantations needed, and have everything Just Work. If we put the >headers only on disk without any way to retrieve them at runtime, we'd >need a different path for kernel self-description in the case where a >running kernel doesn't have an on-disk representation, and that adds a >lot of complexity for everyone everywhere. By providing >guaranteed-correct kernel headers via some runtime interface, we make >a lot of things Just Work, and that has value. Joel specifically is talking about using a module, which *does* have to be in the filesystem. You can't have it both ways, unfortunately. -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
