Quoting Petr Mladek (2021-03-03 02:25:58) > On Mon 2021-03-01 09:47:47, Stephen Boyd wrote: > > The %pS printk format (among some others) is used to print kernel > > addresses symbolically. When the kernel prints an address inside of a > > module, the kernel prints the addresses' symbol name along with the > > module's name that contains the address. Let's make kernel stacktraces > > easier to identify on KALLSYMS builds by including the build ID of a > > module when we print the address. > > > > This is especially helpful for crash debugging with pstore or crashdump > > kernels. If we have the build ID for the module in the stacktrace we can > > request the debug symbols for the module from a remote debuginfod server > > I have read the thread so for. IMHO, all mentioned variants complicate > the logs a lot. Either the backtrace lines are hard to parse. > Or the OOps/panic blocks gets too long when the ID is mentioned > for every loaded module. IMHO, neither proposed solution > is acceptable to be always used. The modules line is already pretty hard to read once it gets beyond 8 or 10 modules. Probably it should be broken up into multiple lines just so it can be read more easily. I find myself having to hunt in there already even without the build ID encoded there. I've also seen folks cut out that line in commit text and when posting to the mailing list because it's just long and noisy already. > > First, I think that only I some developers would actually use > this information. Many of them either know what module was > used or they do not have an easy way to get the debugging > information by the ID. So, it should be configurable > at minimum. It should be configurable because it isn't used or is hard to use? Wouldn't a config variable limit the uptake of this information and further reinforce the fact that nobody will use it? If we always exposed it then maybe we would get new users. I imagine that we could have a robot search crash reports and find the "crashiest" places in the kernel all the way down to the line level and poke kernel developers to look and see why it crashes so much there. If the information is behind a config then the benefit of that analysis will be limited. > > Second, I am not sure that it should be part of each OOps/panic blob. > One solution would be to print the ID by the module loader when > the module gets loaded. It would be mentioned earlier in the log > then. Please no. The kernel log could wrap around before an oops/panic happens and then the ID would be lost. > > Or we could make it available, for example, via /proc. It is a kind > of information that might be gathered later on a rebooted system. > SUSE has "supportconfig" command that allows to gather a lot > of similar information about the system. We use it when > analyzing crashdumps and kernel bugs in general. Please no. The build ID is already available via /sys/module/<modulename>/sections/<sectionname> and /sys/kernel/notes (for vmlinux) but that won't help for panics that reboot. If a panic happens and a new kernel is booted then post processing on the modules and vmlinux could all be incorrect. Furthermore, the modules will have to be found and parsed by some userspace crash processing tool after the reboot. I'd really prefer to rely on the standard build ID vs. a per-distro bespoke solution to finding the information about the binaries the kernel was running. It's just simpler to not need to find out this sort of information about the system when the kernel knows what binary is running already. This is the same reason coredump_filter exists to let coredumping code figure out the build ID of the process that crashes vs. connecting that to some system information daemon. > > Anyway, I consider this a very detailed information that is not > suitable for everyone. It should be availabe on request, like > for example, backtraces from all CPUs, list of tasks, memory info. I suppose I can make a config option for this module printing stuff in the "Modules linked in:" line. Then we can remove it if most distros decide to enable it? I'm slightly disappointed that it can't just be printed all the time for every stacktrace but if there isn't opposition if it's all behind a config option then I will be happy to get 99% of the code upstream.
