Soft resets are fatal just as hard resets, but no reset is "always fatal". There are cases when apps keep working depending on which features are being used. It's still unsafe.
Marek
On Mon, Apr 24, 2023, 03:03 Christian König <christian.koenig@xxxxxxx> wrote:
Am 24.04.23 um 03:43 schrieb André Almeida:
> When a DRM job timeout, the GPU is probably hang and amdgpu have some
> ways to deal with that, ranging from soft recoveries to full device
> reset. Anyway, when userspace ask the kernel the state of the context
> (via AMDGPU_CTX_OP_QUERY_STATE), the kernel reports that the device was
> reset, regardless if a full reset happened or not.
>
> However, amdgpu only marks a context guilty in the ASIC reset path. This
> makes the userspace report incomplete, given that on soft recovery path
> the guilty context is not told that it's the guilty one.
>
> Fix this by marking the context guilty for every type of reset when a
> job timeouts.
The guilty handling is pretty much broken by design and only works
because we go through multiple hops of validating the entity after the
job has already been pushed to the hw.
I think we should probably just remove that completely and use an
approach where we check the in flight submissions in the query state
IOCTL. See my other patch on the mailing list regarding that.
Additional to that I currently didn't considered soft-recovered
submissions as fatal and continue accepting submissions from that
context, but already wanted to talk with Marek about that behavior.
Regards,
Christian.
>
> Signed-off-by: André Almeida <andrealmeid@xxxxxxxxxx>
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 3 ---
> drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 8 +++++++-
> 2 files changed, 7 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> index ac78caa7cba8..ea169d1689e2 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> @@ -4771,9 +4771,6 @@ int amdgpu_device_pre_asic_reset(struct amdgpu_device *adev,
>
> amdgpu_fence_driver_isr_toggle(adev, false);
>
> - if (job && job->vm)
> - drm_sched_increase_karma(&job->base);
> -
> r = amdgpu_reset_prepare_hwcontext(adev, reset_context);
> /* If reset handler not implemented, continue; otherwise return */
> if (r == -ENOSYS)
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> index c3d9d75143f4..097ed8f06865 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> @@ -51,6 +51,13 @@ static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job)
> memset(&ti, 0, sizeof(struct amdgpu_task_info));
> adev->job_hang = true;
>
> + amdgpu_vm_get_task_info(ring->adev, job->pasid, &ti);
> +
> + if (job && job->vm) {
> + DRM_INFO("marking %s context as guilty", ti.process_name);
> + drm_sched_increase_karma(&job->base);
> + }
> +
> if (amdgpu_gpu_recovery &&
> amdgpu_ring_soft_recovery(ring, job->vmid, s_job->s_fence->parent)) {
> DRM_ERROR("ring %s timeout, but soft recovered\n",
> @@ -58,7 +65,6 @@ static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job)
> goto exit;
> }
>
> - amdgpu_vm_get_task_info(ring->adev, job->pasid, &ti);
> DRM_ERROR("ring %s timeout, signaled seq=%u, emitted seq=%u\n",
> job->base.sched->name, atomic_read(&ring->fence_drv.last_seq),
> ring->fence_drv.sync_seq);
