Hi!
As discussed previously the current locking order in TTM of these locks
is bo::reserve -> vm::mmap_sem. This leads to a hack in
the TTM fault() handle to try and revert the locking order. If a
tryreserve failed, we tried to have the vm code release the mmap_sem()
and then schedule, to give the holder of bo::reserve a chance to release
the lock. This solution is no longer legal, since we've been more or
less kindly asked to remove the set_need_resched() call.
Maarten has proposed to invert the locking order. I've previously said I
had no strong preference. The current locking order dates back from the
time when TTM wasn't using unmap_mapping_range() but walked the page
tables itself, updating PTEs as needed. Furthermore it was needed for
user bos that used get_user_pages() in the TTM populate and swap-in
methods. User-bos were removed some time ago but I'm looking at
re-adding them. They would suite the VMware model of cached-only pages
very well. I see uses both in the gallium API, XA's DMA functionality
and openCL.
We would then need a somewhat nicer way to invert the locking order.
I've attached a solution that ups the mmap_sem and then reserves, but
due to how the fault API is done, we then need to release the reserve
and retry the fault. This of course opens up for starvation, but I don't
think starvation at this point is very likely: One thread being refused
to write or read from a buffer object because the GPU is continously
busy with it. If this *would* become a problem, it's probably possible
to modify the fault code to allow us to hold locks until the retried
fault, but that would be a bit invasive, since it touches the arch code....
Basically I'm proposing to keep the current locking order.
/Thomas
diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c
index 1006c15..55c487d 100644
--- a/drivers/gpu/drm/ttm/ttm_bo_vm.c
+++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c
@@ -61,13 +61,22 @@ static int ttm_bo_vm_fault(struct vm_area_struct *vma, struct vm_fault *vmf)
/*
* Work around locking order reversal in fault / nopfn
* between mmap_sem and bo_reserve: Perform a trylock operation
- * for reserve, and if it fails, retry the fault after scheduling.
+ * for reserve, and if it fails, retry the fault after releasing
+ * the mmap_sem and waiting.
*/
ret = ttm_bo_reserve(bo, true, true, false, 0);
if (unlikely(ret != 0)) {
- if (ret == -EBUSY)
- set_need_resched();
+ if (ret == -EBUSY) {
+ if ((vmf->flags & FAULT_FLAG_ALLOW_RETRY) &&
+ !(vmf->flags & FAULT_FLAG_RETRY_NOWAIT)) {
+ up_read(&vma->vm_mm->mmap_sem);
+ ttm_bo_reserve_nolru(bo, true, false, false,
+ NULL);
+ ww_mutex_unlock(&bo->resv->lock);
+ return VM_FAULT_RETRY;
+ }
+ }
return VM_FAULT_NOPAGE;
}
_______________________________________________
dri-devel mailing list
dri-devel@xxxxxxxxxxxxxxxxxxxxx
http://lists.freedesktop.org/mailman/listinfo/dri-devel
