On Wed, Aug 24, 2022 at 04:25:44PM -0400, Peter Xu wrote:
> On Wed, Aug 24, 2022 at 11:56:25AM +1000, Alistair Popple wrote:
> > >> Still I don't know whether there'll be any side effect of having stall tlbs
> > >> in !present ptes because I'm not familiar enough with the private dev swap
> > >> migration code. But I think having them will be safe, even if redundant.
> >
> > What side-effect were you thinking of? I don't see any issue with not
> > TLB flushing stale device-private TLBs prior to the migration because
> > they're not accessible anyway and shouldn't be in any TLB.
>
> Sorry to be misleading, I never meant we must add them. As I said it's
> just that I don't know the code well so I don't know whether it's safe to
> not have it.
>
> IIUC it's about whether having stall system-ram stall tlb in other
> processor would matter or not here. E.g. some none pte that this code
> collected (boosted both "cpages" and "npages" for a none pte) could have
> stall tlb in other cores that makes the page writable there.
For this one, let me give a more detailed example.
It's about whether below could happen:
thread 1 thread 2 thread 3
-------- -------- --------
write to page P (data=P1)
(cached TLB writable)
zap_pte_range()
pgtable lock
clear pte for page P
pgtable unlock
...
migrate_vma_collect
pte none, npages++, cpages++
allocate device page
copy data (with P1)
map pte as device swap
write to page P again
(data updated from P1->P2)
flush tlb
Then at last from processor side P should have data P2 but actually from
device memory it's P1. Data corrupt.
>
> When I said I'm not familiar with the code, it's majorly about one thing I
> never figured out myself, in that migrate_vma_collect_pmd() has this
> optimization to trylock on the page, collect if it succeeded:
>
> /*
> * Optimize for the common case where page is only mapped once
> * in one process. If we can lock the page, then we can safely
> * set up a special migration page table entry now.
> */
> if (trylock_page(page)) {
> ...
> } else {
> put_page(page);
> mpfn = 0;
> }
>
> But it's kind of against a pure "optimization" in that if trylock failed,
> we'll clear the mpfn so the src[i] will be zero at last. Then will we
> directly give up on this page, or will we try to lock_page() again
> somewhere?
>
> The future unmap op is also based on this "cpages", not "npages":
>
> if (args->cpages)
> migrate_vma_unmap(args);
>
> So I never figured out how this code really works. It'll be great if you
> could shed some light to it.
>
> Thanks,
>
> --
> Peter Xu
--
Peter Xu
