On Mon, 20 Jan 2025, Jason Gunthorpe wrote: > On Mon, Jan 20, 2025 at 09:54:15AM +0200, Mike Rapoport wrote: > > Hi, > > > > I'd like to discuss memory persistence across kexec. > > > > Currently there is ongoing work on Kexec HandOver (KHO) [1] that allows > > serialization and deserialization of kernel data as well as preserving > > arbitrary memory ranges across kexec. > > > > In addition, KHO keeps a physically contiguous memory regions that are > > guaranteed to not have any memory that KHO would preserve, but still can be > > used by the system. The kexeced kernel bootstraps itself using those > > regions and sets all handed over memory as in use. KHO users then can > > recover their state from the preserved data. This includes memory > > reservations, where the user can either discard or claim reservations. > > > > KHO can be used as the base layer for implementation of persistence-aware > > memory allocator and persistent in-memory filesystem. > > > > Aside from status update on KHO progress there are a few topics that I would > > like to discuss: > > * Is it feasible and desirable to enable KHO support in tmpfs and hugetlbfs? This is a very timely discussion since the last Linux MM Alignment Session on the topic since some use cases, at least for tmpfs, have emerged. Not necessarily a requirement, but more out of convenience. > > * Or is it better to implement yet another in-memory filesystem dedicated > > for persistence? > > * What is the best way to ensure that the memory we want to persist is not > > scattered all over the place? > > There is alot of talk about taking *drivers* and having them survive > kexec, meaning the driver has to put alot of its state into KHO and > then get it back out again. > > I've been hoping for a model where a driver can be told to "go to KHO" > and the KHO code can be largely contained in the driver and regulated > to recording the driver state. This implies the state may be > fragmented all over memory. > This sounds fantastic if it is doable! > The other direction is that the driver has to start up in some special > KHO mode and KHO becomes invasive on all driver paths to use special > KHO allocations. This seems like a PITA. > > You can see this difference just in the discussion around the iommu > serialization where one idea was to have KHO be an integral (and > invasive!) part of the page table operations from time zero vs some > later serialization at kexec time. > > Regardless, I'm interested in this discussion to bring some > concreteness about how drivers work.. > +1, I'm also interested in this discussion. As previously mentioned[1], we'll also start a biweekly on hypervisor live update to accelerate progress. The first instance of that meeting will be next week, Monday, January 27 at 8am PST (UTC-8). Calendar invites will go out later today for everybody on that email thread, if anybody else is interested in attending on a regular basis please email me. Hoping this can be leveraged as well to build up to LSF/MM/BPF. [1] https://lore.kernel.org/kexec/2908e4ab-abc4-ddd0-b191-fe820856cfb4@xxxxxxxxxx/T/#u
