On Fri, Jul 08, 2022 at 03:03:34PM +0200, Cornelia Huck wrote: > On Mon, Jul 04 2022, Steven Price <steven.price@xxxxxxx> wrote: > > > On 04/07/2022 13:19, Cornelia Huck wrote: > >> On Mon, Jul 04 2022, Steven Price <steven.price@xxxxxxx> wrote: > >> > >>> On 29/06/2022 09:45, Catalin Marinas wrote: > >>>> On Mon, Jun 27, 2022 at 05:55:33PM +0200, Cornelia Huck wrote: > >>> > >>>>> [Postcopy needs a different interface, I guess, so that the migration > >>>>> target can atomically place a received page and its metadata. I see > >>>>> https://lore.kernel.org/all/CAJc+Z1FZxSYB_zJit4+0uTR-88VqQL+-01XNMSEfua-dXDy6Wg@xxxxxxxxxxxxxx/; > >>>>> has there been any follow-up?] > >>>> > >>>> I don't follow the qemu list, so I wasn't even aware of that thread. But > >>>> postcopy, the VMM needs to ensure that both the data and tags are up to > >>>> date before mapping such page into the guest address space. > >>>> > >>> > >>> I'm not sure I see how atomically updating data+tags is different from > >>> the existing issues around atomically updating the data. The VMM needs > >>> to ensure that the guest doesn't see the page before all the data+all > >>> the tags are written. It does mean lazy setting of the tags isn't > >>> possible in the VMM, but I'm not sure that's a worthwhile thing anyway. > >>> Perhaps I'm missing something? > >> > >> For postcopy, we basically want to fault in any not-yet-migrated page > >> via uffd once the guest accesses it. We only get the page data that way, > >> though, not the tag. I'm wondering whether we'd need a 'page+metadata' > >> uffd mode; not sure if that makes sense. Otherwise, we'd need to stop > >> the guest while grabbing the tags for the page as well, and stopping is > >> the thing we want to avoid here. > > > > Ah, I think I see now. UFFDIO_COPY atomically populates the (data) page > > and ensures that no thread will see the partially populated page. But > > there's currently no way of doing that with tags as well. > > Nod. > > > > > I'd not looked at the implementation of userfaultfd before and I'd > > assumed it avoided the need for an 'atomic' operation like this. But > > apparently not! AFAICT either a new ioctl would be needed (which can > > take a tag buffer) or a new flag to UFFDIO_COPY which would tighten the > > alignment requirements of `src` and would copy the tags along with the data. > > I was thinking about a new flag that implies "copy metadata"; not sure > how we would get the same atomicity with a separate ioctl. I've only > just started looking at userfaultfd, though, and I might be on a wrong > track... One thing I'd like to avoid is having something that is too > ARM-specific, I think there are other architecture features that might > have similar issues. Agreed, to propose such an interface we'd better make sure it'll be easily applicable to other similar memory protection mechanisms elsewhere. > > Maybe someone more familiar with uffd and/or postcopy can chime in? Hanving UFFDIO_COPY provide a new flag sounds reasonable to me. I'm curious what's the maximum possible size of the tags and whether they can be embeded already into struct uffdio_copy somehow. Thanks, -- Peter Xu
