+
+VM_BIND feature introduces an optimization where user can
create BO which
+is private to a specified VM via I915_GEM_CREATE_EXT_VM_PRIVATE
flag during
+BO creation. Unlike Shared BOs, these VM private BOs can only
be mapped on
+the VM they are private to and can't be dma-buf exported.
+All private BOs of a VM share the dma-resv object. Hence during
each execbuf
+submission, they need only one dma-resv fence list updated.
Thus, the fast
+path (where required mappings are already bound) submission
latency is O(1)
+w.r.t the number of VM private BOs.
+
+VM_BIND locking hirarchy
+-------------------------
+The locking design here supports the older (execlist based)
execbuf mode, the
+newer VM_BIND mode, the VM_BIND mode with GPU page faults and
possible future
+system allocator support (See `Shared Virtual Memory (SVM) support`_).
+The older execbuf mode and the newer VM_BIND mode without page
faults manages
+residency of backing storage using dma_fence. The VM_BIND mode
with page faults
+and the system allocator support do not use any dma_fence at all.
+
+VM_BIND locking order is as below.
+
+1) Lock-A: A vm_bind mutex will protect vm_bind lists. This
lock is taken in
+ vm_bind/vm_unbind ioctl calls, in the execbuf path and while
releasing the
+ mapping.
+
+ In future, when GPU page faults are supported, we can
potentially use a
+ rwsem instead, so that multiple page fault handlers can take
the read side
+ lock to lookup the mapping and hence can run in parallel.
+ The older execbuf mode of binding do not need this lock.
+
+2) Lock-B: The object's dma-resv lock will protect i915_vma
state and needs to
+ be held while binding/unbinding a vma in the async worker
and while updating
+ dma-resv fence list of an object. Note that private BOs of a
VM will all
+ share a dma-resv object.
+
+ The future system allocator support will use the HMM
prescribed locking
+ instead.
+
+3) Lock-C: Spinlock/s to protect some of the VM's lists like
the list of
+ invalidated vmas (due to eviction and userptr invalidation) etc.
+
+When GPU page faults are supported, the execbuf path do not
take any of these
+locks. There we will simply smash the new batch buffer address
into the ring and
+then tell the scheduler run that. The lock taking only happens
from the page
+fault handler, where we take lock-A in read mode, whichever
lock-B we need to
+find the backing storage (dma_resv lock for gem objects, and
hmm/core mm for
+system allocator) and some additional locks (lock-D) for taking
care of page
+table races. Page fault mode should not need to ever manipulate
the vm lists,
+so won't ever need lock-C.
+
+VM_BIND LRU handling
+---------------------
+We need to ensure VM_BIND mapped objects are properly LRU
tagged to avoid
+performance degradation. We will also need support for bulk LRU
movement of
+VM_BIND objects to avoid additional latencies in execbuf path.
+
+The page table pages are similar to VM_BIND mapped objects (See
+`Evictable page table allocations`_) and are maintained per VM
and needs to
+be pinned in memory when VM is made active (ie., upon an
execbuf call with
+that VM). So, bulk LRU movement of page table pages is also needed.
+
+VM_BIND dma_resv usage
+-----------------------
+Fences needs to be added to all VM_BIND mapped objects. During
each execbuf
+submission, they are added with DMA_RESV_USAGE_BOOKKEEP usage
to prevent
+over sync (See enum dma_resv_usage). One can override it with either
+DMA_RESV_USAGE_READ or DMA_RESV_USAGE_WRITE usage during object
dependency
+setting (either through explicit or implicit mechanism).
+
+When vm_bind is called for a non-private object while the VM is already
+active, the fences need to be copied from VM's shared dma-resv object
+(common to all private objects of the VM) to this non-private object.
+If this results in performance degradation, then some optimization will
+be needed here. This is not a problem for VM's private objects
as they use
+shared dma-resv object which is always updated on each execbuf
submission.
+
+Also, in VM_BIND mode, use dma-resv apis for determining object
activeness
+(See dma_resv_test_signaled() and dma_resv_wait_timeout()) and
do not use the
+older i915_vma active reference tracking which is deprecated.
This should be
+easier to get it working with the current TTM backend.
+
+Mesa use case
+--------------
+VM_BIND can potentially reduce the CPU overhead in Mesa (both
Vulkan and Iris),
+hence improving performance of CPU-bound applications. It also
allows us to
+implement Vulkan's Sparse Resources. With increasing GPU
hardware performance,
+reducing CPU overhead becomes more impactful.
+
+
+Other VM_BIND use cases
+========================
+
+Long running Compute contexts
+------------------------------
+Usage of dma-fence expects that they complete in reasonable
amount of time.
+Compute on the other hand can be long running. Hence it is
appropriate for
+compute to use user/memory fence (See `User/Memory Fence`_) and
dma-fence usage
+must be limited to in-kernel consumption only.
+
+Where GPU page faults are not available, kernel driver upon
buffer invalidation
+will initiate a suspend (preemption) of long running context,
finish the
+invalidation, revalidate the BO and then resume the compute
context. This is
+done by having a per-context preempt fence which is enabled
when someone tries
+to wait on it and triggers the context preemption.
+
+User/Memory Fence
+~~~~~~~~~~~~~~~~~~
+User/Memory fence is a <address, value> pair. To signal the
user fence, the
+specified value will be written at the specified virtual
address and wakeup the
+waiting process. User fence can be signaled either by the GPU
or kernel async
+worker (like upon bind completion). User can wait on a user
fence with a new
+user fence wait ioctl.
+
+Here is some prior work on this:
+https://patchwork.freedesktop.org/patch/349417/
+
+Low Latency Submission
+~~~~~~~~~~~~~~~~~~~~~~~
+Allows compute UMD to directly submit GPU jobs instead of
through execbuf
+ioctl. This is made possible by VM_BIND is not being
synchronized against
+execbuf. VM_BIND allows bind/unbind of mappings required for
the directly
+submitted jobs.
+
+Debugger
+---------
+With debug event interface user space process (debugger) is
able to keep track
+of and act upon resources created by another process (debugged)
and attached
+to GPU via vm_bind interface.
+
+GPU page faults
+----------------
+GPU page faults when supported (in future), will only be
supported in the
+VM_BIND mode. While both the older execbuf mode and the newer
VM_BIND mode of
+binding will require using dma-fence to ensure residency, the
GPU page faults
+mode when supported, will not use any dma-fence as residency is
purely managed
+by installing and removing/invalidating page table entries.
+
+Page level hints settings
+--------------------------
+VM_BIND allows any hints setting per mapping instead of per BO.
+Possible hints include read-only mapping, placement and atomicity.
+Sub-BO level placement hint will be even more relevant with
+upcoming GPU on-demand page fault support.
+
+Page level Cache/CLOS settings
+-------------------------------
+VM_BIND allows cache/CLOS settings per mapping instead of per BO.
+
+Evictable page table allocations
+---------------------------------
+Make pagetable allocations evictable and manage them similar to VM_BIND
+mapped objects. Page table pages are similar to persistent
mappings of a
+VM (difference here are that the page table pages will not have
an i915_vma
+structure and after swapping pages back in, parent page link
needs to be
+updated).
+
+Shared Virtual Memory (SVM) support
+------------------------------------
+VM_BIND interface can be used to map system memory directly
(without gem BO
+abstraction) using the HMM interface. SVM is only supported
with GPU page
+faults enabled.
+
+VM_BIND UAPI
+=============
+
+.. kernel-doc:: Documentation/gpu/rfc/i915_vm_bind.h
diff --git a/Documentation/gpu/rfc/index.rst
b/Documentation/gpu/rfc/index.rst
index 91e93a705230..7d10c36b268d 100644
--- a/Documentation/gpu/rfc/index.rst
+++ b/Documentation/gpu/rfc/index.rst
@@ -23,3 +23,7 @@ host such documentation:
.. toctree::
i915_scheduler.rst
+
+.. toctree::
+
+ i915_vm_bind.rst
