Hi, On Wed, 23 Jun 2021 at 17:20, Daniel Vetter <daniel.vetter@xxxxxxxx> wrote: > + * > + * IMPLICIT SYNCHRONIZATION RULES: > + * > + * Drivers which support implicit synchronization of buffer access as > + * e.g. exposed in `Implicit Fence Poll Support`_ should follow the > + * below rules. 'Should' ... ? Must. > + * - Drivers should add a shared fence through > + * dma_resv_add_shared_fence() for anything the userspace API > + * considers a read access. This highly depends upon the API and > + * window system: E.g. OpenGL is generally implicitly synchronized on > + * Linux, but explicitly synchronized on Android. Whereas Vulkan is > + * generally explicitly synchronized for everything, and window system > + * buffers have explicit API calls (which then need to make sure the > + * implicit fences store here in @resv are updated correctly). > + * > + * - [...] Mmm, I think this is all right, but it could be worded much more clearly. Right now it's a bunch of points all smashed into one, and there's a lot of room for misinterpretation. Here's a strawman, starting with most basic and restrictive, working through to when you're allowed to wriggle your way out: Rule 1: Drivers must add a shared fence through dma_resv_add_shared_fence() for any read accesses against that buffer. This appends a fence to the shared array, ensuring that any future non-read access will be synchronised against this operation to only begin after it has completed. Rule 2: Drivers must add an exclusive fence through dma_resv_add_excl_fence() for any write accesses against that buffer. This replaces the exclusive fence with the new operation, ensuring that all future access will be synchronised against this operation to only begin after it has completed. Rule 3: Drivers must synchronise all accesses to buffers against existing implicit fences. Read accesses must synchronise against the exclusive fence (read-after-write), and write accesses must synchronise against both the exclusive (write-after-write) and shared (write-after-read) fences. Note 1: Users like OpenGL and window systems on non-Android userspace are generally implicitly synchronised. An implicitly-synchronised userspace is unaware of fences from prior operations, so the kernel mediates scheduling to create the illusion that GPU work is FIFO. For example, an application will flush and schedule GPU write work to render its image, then immediately tell the window system to display that image; the window system may immediately flush and schedule GPU read work to display that image, with neither waiting for the write to have completed. The kernel provides coherence by synchronising the read access against the write fence in the exclusive slot, so that the image displayed is correct. Note 2: Users like Vulkan and Android window system are generally explicitly synchronised. An explicitly-synchronised userspace is responsible for tracking its own read and write access and providing the kernel with synchronisation barriers. For instance, a Vulkan application rendering to a buffer and subsequently using it as a read texture, must annotate the read operation with a read-after-write synchronisation barrier. Note 3: Implicit and explicit userspace can coexist. For instance, an explicitly-synchronised Vulkan application may be running as a client of an implicitly-synchronised window system which uses OpenGL for composition; an implicitly-synchronised OpenGL application may be running as a client of a window system which uses Vulkan for composition. Note 4: Some subsystems, for example V4L2, do not pipeline operations, and instead only return to userspace when the scheduled work against a buffer has fully retired. Exemption 1: Fully self-coherent userspace may skip implicit synchronisation barriers. For instance, accesses between two Vulkan-internal buffers allocated by a single application do not need to synchronise against each other's implicit fences, as the client is responsible for explicitly providing barriers for access. A self-contained OpenGL userspace also has no need to implicitly synchronise its access if the driver instead tracks all access and inserts the appropriate synchronisation barriers. Exemption 2: When implicit and explicit userspace coexist, the explicit side may skip intermediate synchronisation, and only place synchronisation barriers at transition points. For example, a Vulkan compositor displaying a buffer from an OpenGL application would need to synchronise its first access against the fence placed in the exclusive implicit-synchronisation slot. Once this read has fully retired, the compositor has no need to participate in implicit synchronisation until it is ready to return the buffer to the application, at which point it must insert all its non-retired accesses into the shared slot, which the application will then synchronise future write accesses against. Cheers, Daniel
