This is different than Mike's "memory persistence over kexec" topic which has to do with Kexec Hand Over (KHO). This is about tracing and generic pstore itself. Here's the problem statement: To enable tracing over persistent memory in the field we add to the kernel command line: reserve_mem=20M:12M:trace trace_instance=boot_mapped^traceoff@trace The "reserve_mem=20M:12M:trace" will reserve 20 megabytes of memory that is aligned by 12 megabytes and label it "trace". The "trace_instance=boot_mapped^traceoff@trace" will look for memory from the "reserve_mem" that has the label "trace", assign it to the boot_mapped instance (found in /sys/kernel/tracing/instances/boot_mapped) and it will be have tracing disabled by default on boot. (Note that pstore can also get its persistent memory from the "reserve_mem" option as well). On boot up, various events will be started in the "boot_mapped" trace instance. If there's a crash, on the next boot, the "boot_mapped" trace instance will be read which will contain all the events that led up to the crash (this can also trace reboots too). Obviously this depends on the machine not clearing memory and only doing a soft reset where the memory is still persistent from the last boot. This also only works if the "reserve_mem=20M:12M:trace" reserves the memory in the same location as it did from the previous boot. Hence the problem I have here. Namely KASLR. If I disable KASLR (nokaslr on the kernel command line) then I found that the memory reserved is consistent between boots and I have not hit any issues while debugging in the lab, or on my own devices, as I can safely add "nokaslr" on test machines. But for productions machines and for machines in the field, that is not an option. KASLR can move the placement of the kernel where it overlaps the previous saved buffer, or what I found is even more common, it causes something that was reserved before the "reserve_mem" was called to be allocated in a different location which caused the memory reserved by reserve_mem to move. I also found (at least on a few machines I played with), adding a large alignment (like 12M) helps with that from happening. That is, things that were allocated before "reserve_mem" was parsed can move around and the memory reserved stays consistent. That helps, but does not prevent it. My question: Would it be possible to have some communication between KASLR and the kernel to know about this reserved memory and work to keep it from moving? Since movement of allocations that happen before "reserve_mem" is parsed can affect where the reserved memory is allocated, I'm not sure just keeping KASLR away from the reserved memory is enough. Perhaps, after a successful allocation, there can be some memory range that can communicate back to KASLR on the next boot to actually reserve that memory location again, and then communicate back to the booting kernel that this memory was reserved there and the "reserve_mem" will just used that memory? Really, I'm looking for any ideas that can help keep "reserve_mem" allocating memory at the same location after each boot. Thoughts? -- Steve
