On Mon, Jan 20, 2025 at 8:54 AM Mike Rapoport <rppt@xxxxxxxxxx> wrote:
>
> Hi,
>
> I'd like to discuss memory persistence across kexec.
>
Hi, I'm very interested in this topic as well, I'd like to join the club )
> Currently there is ongoing work on Kexec HandOver (KHO) [1] that allows
> serialization and deserialization of kernel data as well as preserving
> arbitrary memory ranges across kexec.
>
To be able to perform live update of hypervisor kernel with running
VMs which use VFIO devices
we would need to [de]serialize a lots of different and complex states
(PCI, IOMMU, VFIO ...)
When I've looked at KHO I found that the process of describing data
using KHO is complicated, requires
to write a lot of code that needs to be invaded deeply into subsystem
code. So I think this might be
a blocker for applying KHO to VFIO device state which is more
complicated than the ftrace buffers.
To address this particular issue I've come up with the proof of
concept which I sent a few months ago:
https://lkml.kernel.org/r/20241002160722.20025-1-arbn@xxxxxxxxxxxxxxx
The idea behind was inspired by QEMU's VMSTATE mechanism which solves
similar problem -
to describe and migrate devices states across different instances of QEMU.
As an example, I've chosen to preserve ftrace buffers as well, so it's
easier to compare with KHO approach.
> In addition, KHO keeps a physically contiguous memory regions that are
> guaranteed to not have any memory that KHO would preserve, but still can be
> used by the system. The kexeced kernel bootstraps itself using those
> regions and sets all handed over memory as in use. KHO users then can
> recover their state from the preserved data. This includes memory
> reservations, where the user can either discard or claim reservations.
>
> KHO can be used as the base layer for implementation of persistence-aware
> memory allocator and persistent in-memory filesystem.
>
> Aside from status update on KHO progress there are a few topics that I would
> like to discuss:
> * Is it feasible and desirable to enable KHO support in tmpfs and hugetlbfs?
> * Or is it better to implement yet another in-memory filesystem dedicated
> for persistence?
We would definitely need a framework to [de]serialize data. With that
we should be able
to preserve tmpfs/hugetblfs (and it probably will be easier than
preserving some device state).
So yet another in-memory filesystem should come only as a solution for
some potential problem, just for example:
- serialization of tmpfs/hugetlbfs requires unreasonable amount of
memory (or time to process)
- implementation ends up too complicated and fragile, so it's just
better to have separate dedicated fs
- whatever else comes up...
