On Thu, Jun 24, 2021 at 1:08 PM Daniel Stone <daniel@xxxxxxxxxxxxx> wrote: > > Hi, > > On Wed, 23 Jun 2021 at 17:20, Daniel Vetter <daniel.vetter@xxxxxxxx> wrote: > > + * > > + * IMPLICIT SYNCHRONIZATION RULES: > > + * > > + * Drivers which support implicit synchronization of buffer access as > > + * e.g. exposed in `Implicit Fence Poll Support`_ should follow the > > + * below rules. > > 'Should' ... ? Must. Yeah I guess I can upgrade a bunch of them. > > + * - Drivers should add a shared fence through > > + * dma_resv_add_shared_fence() for anything the userspace API > > + * considers a read access. This highly depends upon the API and > > + * window system: E.g. OpenGL is generally implicitly synchronized on > > + * Linux, but explicitly synchronized on Android. Whereas Vulkan is > > + * generally explicitly synchronized for everything, and window system > > + * buffers have explicit API calls (which then need to make sure the > > + * implicit fences store here in @resv are updated correctly). > > + * > > + * - [...] > > Mmm, I think this is all right, but it could be worded much more > clearly. Right now it's a bunch of points all smashed into one, and > there's a lot of room for misinterpretation. > > Here's a strawman, starting with most basic and restrictive, working > through to when you're allowed to wriggle your way out: > > Rule 1: Drivers must add a shared fence through > dma_resv_add_shared_fence() for any read accesses against that buffer. > This appends a fence to the shared array, ensuring that any future > non-read access will be synchronised against this operation to only > begin after it has completed. > > Rule 2: Drivers must add an exclusive fence through > dma_resv_add_excl_fence() for any write accesses against that buffer. > This replaces the exclusive fence with the new operation, ensuring > that all future access will be synchronised against this operation to > only begin after it has completed. > > Rule 3: Drivers must synchronise all accesses to buffers against > existing implicit fences. Read accesses must synchronise against the > exclusive fence (read-after-write), and write accesses must > synchronise against both the exclusive (write-after-write) and shared > (write-after-read) fences. > > Note 1: Users like OpenGL and window systems on non-Android userspace > are generally implicitly synchronised. An implicitly-synchronised > userspace is unaware of fences from prior operations, so the kernel > mediates scheduling to create the illusion that GPU work is FIFO. For > example, an application will flush and schedule GPU write work to > render its image, then immediately tell the window system to display > that image; the window system may immediately flush and schedule GPU > read work to display that image, with neither waiting for the write to > have completed. The kernel provides coherence by synchronising the > read access against the write fence in the exclusive slot, so that the > image displayed is correct. > > Note 2: Users like Vulkan and Android window system are generally > explicitly synchronised. An explicitly-synchronised userspace is > responsible for tracking its own read and write access and providing > the kernel with synchronisation barriers. For instance, a Vulkan > application rendering to a buffer and subsequently using it as a read > texture, must annotate the read operation with a read-after-write > synchronisation barrier. > > Note 3: Implicit and explicit userspace can coexist. For instance, an > explicitly-synchronised Vulkan application may be running as a client > of an implicitly-synchronised window system which uses OpenGL for > composition; an implicitly-synchronised OpenGL application may be > running as a client of a window system which uses Vulkan for > composition. > > Note 4: Some subsystems, for example V4L2, do not pipeline operations, > and instead only return to userspace when the scheduled work against a > buffer has fully retired. > > Exemption 1: Fully self-coherent userspace may skip implicit > synchronisation barriers. For instance, accesses between two > Vulkan-internal buffers allocated by a single application do not need > to synchronise against each other's implicit fences, as the client is > responsible for explicitly providing barriers for access. A > self-contained OpenGL userspace also has no need to implicitly > synchronise its access if the driver instead tracks all access and > inserts the appropriate synchronisation barriers. > > Exemption 2: When implicit and explicit userspace coexist, the > explicit side may skip intermediate synchronisation, and only place > synchronisation barriers at transition points. For example, a Vulkan > compositor displaying a buffer from an OpenGL application would need > to synchronise its first access against the fence placed in the > exclusive implicit-synchronisation slot. Once this read has fully > retired, the compositor has no need to participate in implicit > synchronisation until it is ready to return the buffer to the > application, at which point it must insert all its non-retired > accesses into the shared slot, which the application will then > synchronise future write accesses against. So I think this is excellent, but maybe better suited in the uapi section as a sperate chapter? Essentially keep your rules in the driver-internal docs, but move the Note/excemptions into the uapi section under a "Implicit Sync Mode of Operation" or whatever heading? The other thing to keep in mind is that this is very much incomplete: I'm silent on what drivers should do exactly with these fences. That's largely because I haven't fully completed that audit, and there's a pile of bugs there still. -Daniel -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ Intel-gfx mailing list Intel-gfx@xxxxxxxxxxxxxxxxxxxxx https://lists.freedesktop.org/mailman/listinfo/intel-gfx
