On Mon, May 25, 2020 at 09:51:30AM -0400, Andrey Grodzovsky wrote: > On 5/25/20 8:46 AM, Pekka Paalanen wrote: > > > From: Pekka Paalanen <pekka.paalanen@xxxxxxxxxxxxx> > > > > Set up the expectations on how hot-unplugging a DRM device should look like to > > userspace. > > > > Written by Daniel Vetter's request and largely based on his comments in IRC and > > from https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Farchives%2Fdri-devel%2F2020-May%2F265484.html&data=02%7C01%7Candrey.grodzovsky%40amd.com%7Cc9676f35bbdf4d5a052808d800a9b517%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637260076178891269&sdata=tbOTr7TfESohEgWspomM1sbMq4U4n7bOvdS6JlYifmM%3D&reserved=0 . > > > > Signed-off-by: Pekka Paalanen <pekka.paalanen@xxxxxxxxxxxxx> > > Cc: Daniel Vetter <daniel@xxxxxxxx> > > Cc: Andrey Grodzovsky <andrey.grodzovsky@xxxxxxx> > > Cc: Dave Airlie <airlied@xxxxxxxxxx> > > Cc: Sean Paul <sean@xxxxxxxxxx> > > Cc: Simon Ser <contact@xxxxxxxxxxx> > > > > --- > > > > v2: > > - mmap reads/writes undefined (danvet) > > - make render ioctl behaviour driver-specific (danvet) > > - restructure the mmap paragraphs (danvet) > > - chardev minor notes (Simon) > > - open behaviour (danvet) > > - DRM leasing behaviour (danvet) > > - added links > > > > Disclaimer: I am a userspace developer writing for other userspace developers. > > I took some liberties in defining what should happen without knowing what is > > actually possible or what existing drivers already implement. > > --- > > Documentation/gpu/drm-uapi.rst | 102 +++++++++++++++++++++++++++++++++ > > 1 file changed, 102 insertions(+) > > > > diff --git a/Documentation/gpu/drm-uapi.rst b/Documentation/gpu/drm-uapi.rst > > index 56fec6ed1ad8..520b8e640ad1 100644 > > --- a/Documentation/gpu/drm-uapi.rst > > +++ b/Documentation/gpu/drm-uapi.rst > > @@ -1,3 +1,5 @@ > > +.. Copyright 2020 DisplayLink (UK) Ltd. > > + > > =================== > > Userland interfaces > > =================== > > @@ -162,6 +164,106 @@ other hand, a driver requires shared state between clients which is > > visible to user-space and accessible beyond open-file boundaries, they > > cannot support render nodes. > > +Device Hot-Unplug > > +================= > > + > > +.. note:: > > + The following is the plan. Implementation is not there yet > > + (2020 May). > > + > > +Graphics devices (display and/or render) may be connected via USB (e.g. > > +display adapters or docking stations) or Thunderbolt (e.g. eGPU). An end > > +user is able to hot-unplug this kind of devices while they are being > > +used, and expects that the very least the machine does not crash. Any > > +damage from hot-unplugging a DRM device needs to be limited as much as > > +possible and userspace must be given the chance to handle it if it wants > > +to. Ideally, unplugging a DRM device still lets a desktop to continue > > +running, but that is going to need explicit support throughout the whole > > +graphics stack: from kernel and userspace drivers, through display > > +servers, via window system protocols, and in applications and libraries. > > So to support all the requirements in this document only kernel changes > should be enough and no changes are required from user mode part of the > stack ? > > > + > > +Other scenarios that should lead to the same are: unrecoverable GPU > > +crash, PCI device disappearing off the bus, or forced unbind of a driver > > +from the physical device. > > + > > +In other words, from userspace perspective everything needs to keep on > > +working more or less, until userspace stops using the disappeared DRM > > +device and closes it completely. Userspace will learn of the device > > +disappearance from the device removed uevent > > > Is this uevent already implemented ? Can you point me to the code ? > > > > or in some cases > > +driver-specific ioctls returning EIO. > > + > > +Only after userspace has closed all relevant DRM device and dmabuf file > > +descriptors and removed all mmaps, the DRM driver can tear down its > > +instance for the device that no longer exists. If the same physical > > +device somehow comes back in the mean time, it shall be a new DRM > > +device. > > + > > +Similar to PIDs, chardev minor numbers are not recycled immediately. A > > +new DRM device always picks the next free minor number compared to the > > +previous one allocated, and wraps around when minor numbers are > > +exhausted. > > + > > +Requirements for UAPI > > +--------------------- > > + > > +The goal raises at least the following requirements for the kernel and > > +drivers: > > + > > +- The kernel must not hang, crash or oops, no matter what userspace was > > + in the middle of doing when the device disappeared. > > + > > +- All GPU jobs that can no longer run must have their fences > > + force-signalled to avoid inflicting hangs to userspace. > > + > > +- KMS connectors must change their status to disconnected. > > + > > +- Legacy modesets and pageflips fake success. > > + > > +- Atomic commits, both real and TEST_ONLY, fake success. > > + > > +- Pending non-blocking KMS operations deliver the DRM events userspace > > + is expecting. > > > The 4 points above refer to mode setting/display attached card and are > irrelevant for secondary GPU (e.g. DRI-PRIME scenario) or no display system > in general. Maybe we can somehow highlight this in the document and I on the > implementing side can then decide as a first step to concentrate on > implementing the non display case as a first step or the only step. In > general and correct me if I am wrong, render only GPUs (or compute only) are > the majority of cases where you would want to be able to detach/attach GPU > on the fly (e.g attach stronger secondary graphic card to a laptop to get > high performance in a game or add/remove a GPU to/from a compute cluster) Yeah maybe splitting this up into kms section, and rendering/cross driver section (the dma-buf/fence stuff is relevant for both display and rendering) would make some sense. -Daniel > > Andrey > > > > + > > +- dmabuf which point to memory that has disappeared will continue to > > + be successfully imported if it would have succeeded before the > > + disappearance. > > + > > +- Attempting to import a dmabuf to a disappeared device will succeed if > > + it would have succeeded without the disappearance. > > + > > +- Some userspace APIs already define what should happen when the device > > + disappears (OpenGL, GL ES: `GL_KHR_robustness`_; `Vulkan`_: > > + VK_ERROR_DEVICE_LOST; etc.). DRM drivers are free to implement this > > + behaviour the way they see best, e.g. returning failures in > > + driver-specific ioctls and handling those in userspace drivers, or > > + rely on uevents, and so on. > > + > > +- open() on a device node whose underlying device has disappeared will > > + fail. > > + > > +- Attempting to create a DRM lease on a disappeared DRM device will > > + fail. Existing DRM leases remain. > > + > > +.. _GL_KHR_robustness: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.khronos.org%2Fregistry%2FOpenGL%2Fextensions%2FKHR%2FKHR_robustness.txt&data=02%7C01%7Candrey.grodzovsky%40amd.com%7Cc9676f35bbdf4d5a052808d800a9b517%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637260076178891269&sdata=m%2FneRusoe6qGVU8Edk%2FncaD7eSJZXtPnA1IqLr7k%2Bos%3D&reserved=0 > > +.. _Vulkan: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.khronos.org%2Fvulkan%2F&data=02%7C01%7Candrey.grodzovsky%40amd.com%7Cc9676f35bbdf4d5a052808d800a9b517%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637260076178901265&sdata=WsfLduUBzRKlybOJb5PQViBWYu5DgleEeycmf76l3UU%3D&reserved=0 > > + > > +Requirements for memory maps > > +---------------------------- > > + > > +Memory maps have further requirements. If the underlying memory > > +disappears, the mmap is modified such that reads and writes will still > > +complete successfully but the result is undefined. This applies to both > > +userspace mmap()'d memory and memory pointed to by dmabuf which might be > > +mapped to other devices. > > + > > +Raising SIGBUS is not an option, because userspace cannot realistically > > +handle it. Signal handlers are global, which makes them extremely > > +difficult to use correctly from libraries like those that Mesa produces. > > +Signal handlers are not composable, you can't have different handlers > > +for GPU1 and GPU2 from different vendors, and a third handler for > > +mmapped regular files. Threads cause additional pain with signal > > +handling as well. > > + > > .. _drm_driver_ioctl: > > IOCTL Support on Device Nodes -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ dri-devel mailing list dri-devel@xxxxxxxxxxxxxxxxxxxxx https://lists.freedesktop.org/mailman/listinfo/dri-devel
