On Thu, Jan 16, 2025 at 12:32 PM Peter Xu <peterx@xxxxxxxxxx> wrote: > > On Thu, Jan 16, 2025 at 03:19:49PM -0500, Peter Xu wrote: > > James, > > > > Sorry for a late reply. > > > > I still do have one or two pure questions, but nothing directly relevant to > > your series. > > > > On Thu, Jan 02, 2025 at 12:53:11PM -0500, James Houghton wrote: > > > So I'm not pushing for KVM Userfault to replace userfaultfd; it's not > > > worth the extra/duplicated complexity. And at LPC, Paolo and Sean > > > indicated that this direction was indeed wrong. I have another way to > > > make this work in mind. :) > > > > Do you still want to share it, more or less? :) I think I'm referring to how to make 4K demand fetches for 1G-backed guest memory work, and I kind of said what I was thinking a little further down: On Thu, Jan 2, 2025 at 9:53 AM James Houghton <jthoughton@xxxxxxxxxx> wrote: > > FWIW, I think userspace mapping of gmem + userfaultfd support for > userspace-mapped gmem + 1G page support for gmem = good 1G post-copy > for QEMU (i.e., use gmem instead of hugetlbfs after gmem supports 1G > pages). > > Remember the feedback I got from LSFMM a while ago? "don't use > hugetlbfs." gmem seems like the natural replacement. I guess this might not work if QEMU *needs* to use HugeTLB for whatever reason, but Google's hypervisor just needs 1G pages; it doesn't matter where they come from really. > > > For the gmem case, userfaultfd cannot be used, so KVM Userfault isn't > > > replacing it. And as of right now anyway, KVM Userfault *does* provide > > > a complete post-copy system for gmem. > > > > > > When gmem pages can be mapped into userspace, for post-copy to remain > > > functional, userspace-mapped gmem will need userfaultfd integration. > > > Keep in mind that even after this integration happens, userfaultfd > > > alone will *not* be a complete post-copy solution, as vCPU faults > > > won't be resolved via the userspace page tables. > > > > Do you know in context of CoCo, whether a private page can be accessed at > > all outside of KVM? > > > > I think I'm pretty sure now a private page can never be mapped to > > userspace. However, can another module like vhost-kernel access it during > > postcopy? My impression of that is still a yes, but then how about > > vhost-user? > > > > Here, the "vhost-kernel" part represents a question on whether private > > pages can be accessed at all outside KVM. While "vhost-user" part > > represents a question on whether, if the previous vhost-kernel question > > answers as "yes it can", such access attempt can happen in another > > process/task (hence, not only does it lack KVM context, but also not > > sharing the same task context). > > Right after I sent it, I just recalled whenever a device needs to access > the page, it needs to be converted to shared pages first.. Yep! This is my understanding anyway. Devices will need to GUP or use the userspace page tables to access guest memory; both of which will go to userfaultfd. And userspace hasn't told KVM to make some pages shared, then these GUPs/faults will fail. Maybe Trusted I/O changes some things here... let me reply to Sean. :) > So I suppose the questions were not valid at all! It is not about the > context but that the pages will be shared always whenever a device in > whatever form will access it.. > > Fundamentally I'm thinking about whether userfaultfd must support (fd, > offset) tuple. Now I suppose it's not, because vCPUs accessing > private/shared will all exit to userspace, while all non-vCPU / devices can > access shared pages only. > > In that case, looks like userfaultfd can support CoCo on device emulations > by sticking with virtual-address traps like before, at least from that > specific POV. Yeah, I don't think the userfaultfd API needs to change to support gmem, because it's going to be using the VMAs / user mappings of gmem.
