On 11/15/19 6:06 AM, Jason Gunthorpe wrote:
On Thu, Nov 14, 2019 at 03:06:05PM -0800, Ralph Campbell wrote:
On 11/13/19 5:51 AM, Christoph Hellwig wrote:
On Tue, Nov 12, 2019 at 11:45:52PM +0000, Jason Gunthorpe wrote:
Well, it would mean registering for the whole process address space.
I'll give it a try.
I'm not sure it makes much sense that this testing is essentially
modeled after nouveau's usage which is very strange compared to the
other drivers.
Which means we really should make the test cases fit the proper usage.
Maybe defer the tests for 5.5 and just merge the first patch for now?
I think this a good point to discuss.
Some devices will want to register for all changes to the process address
space because there is no requirement to preregister regions that the
device can access verses devices like InfiniBand where a range of addresses
have to be registered before the device can access those addresses.
But this is a very bad idea to register and do HW actions for ranges
that can't possibly have any pages registered. It slows down the
entire application
I think the ODP approach might be saner, when it mirrors the entire
address space it chops it up into VA chunks, and once a page is
registered on the HW the VA chunk goes into the interval tree.
Presumably the GPU also has some kind of page table tree and you could
set one of the levels as the VA interval when there are populated children
Jason
I wasn't suggesting that HW invalidates happen in two places.
I'm suggesting the two styles of invalidates can work together.
For example, what if a driver calls mmu_notifier_register(mn, mm)
to register for address space wide invalidations, then some time
later there is a device page table fault and the driver calls
mmu_range_notifier_insert() but with a NULL ops.invalidate.
The fault handler follows the nouveau/test_hmm pattern to call
mmu_range_read_begin()
hmm_range_fault()
device lock
mmu_range_read_retry()
update device page tables
device unlock
mmu_range_notifier_remove()
The global invalidate() callback would get the device lock and
call into mm to update the sequence number of any affected ranges
instead of having a range invalidate callback, and then do the HW
invalidations.
