|
Am 2019-04-30 um 1:03 p.m. schrieb
Koenig, Christian:
The only real solution I can see is to be able to reliable kill shaders in an OOM situation.Well, we can in fact preempt our compute shaders with low latency. Killing a KFD process will do exactly that.I've taken a look at that thing as well and to be honest it is not even remotely sufficient. We need something which stops the hardware *immediately* from accessing system memory, and not wait for the SQ to kill all waves, flush caches etc... One possibility I'm playing around with for a while is to replace the root PD for the VMIDs in question on the fly. E.g. we just let it point to some dummy which redirects everything into nirvana. But implementing this is easier said than done... Warming up this thread, since I just fixed another bug that was enabled by artificial memory pressure due to the GTT limit. I think disabling the PD for the VMIDs is a good idea. A problem is that HWS firmware updates PD pointers in the background for its VMIDs. So this would require a reliable and fast way to kill the HWS first. An alternative I thought about is, disabling bus access at the
BIF level if that's possible somehow. Basically we would
instantaneously kill all GPU system memory access, signal all
fences or just remove all fences from all BO reservations
(reservation_object_add_excl_fence(resv, NULL)) to allow memory to
be freed, let the OOM killer do its thing, and when the dust
settles, reset the GPU. Regards, Regards, Christian. |
Attachment:
signature.asc
Description: OpenPGP digital signature
_______________________________________________ amd-gfx mailing list amd-gfx@xxxxxxxxxxxxxxxxxxxxx https://lists.freedesktop.org/mailman/listinfo/amd-gfx
