On Mon, Aug 23, 2021 at 3:59 AM Quan, Evan <Evan.Quan@xxxxxxx> wrote:
>
> [AMD Official Use Only]
>
>
>
> > -----Original Message-----
> > From: Alex Deucher <alexdeucher@xxxxxxxxx>
> > Sent: Friday, August 20, 2021 10:23 PM
> > To: Quan, Evan <Evan.Quan@xxxxxxx>
> > Cc: Lazar, Lijo <Lijo.Lazar@xxxxxxx>; Zhu, James <James.Zhu@xxxxxxx>;
> > amd-gfx@xxxxxxxxxxxxxxxxxxxxx; Liu, Leo <Leo.Liu@xxxxxxx>; Deucher,
> > Alexander <Alexander.Deucher@xxxxxxx>; Chen, Guchun
> > <Guchun.Chen@xxxxxxx>; Pan, Xinhui <Xinhui.Pan@xxxxxxx>
> > Subject: Re: [PATCH] drm/amdgpu: add missing cleanups for Polaris12
> > UVD/VCE on suspend
> >
> > On Thu, Aug 19, 2021 at 10:15 PM Quan, Evan <Evan.Quan@xxxxxxx> wrote:
> > >
> > > [AMD Official Use Only]
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > From: Lazar, Lijo <Lijo.Lazar@xxxxxxx>
> > > Sent: Thursday, August 19, 2021 10:36 PM
> > > To: Zhu, James <James.Zhu@xxxxxxx>; Quan, Evan
> > <Evan.Quan@xxxxxxx>;
> > > amd-gfx@xxxxxxxxxxxxxxxxxxxxx
> > > Cc: Deucher, Alexander <Alexander.Deucher@xxxxxxx>; Chen, Guchun
> > > <Guchun.Chen@xxxxxxx>; Pan, Xinhui <Xinhui.Pan@xxxxxxx>
> > > Subject: RE: [PATCH] drm/amdgpu: add missing cleanups for Polaris12
> > > UVD/VCE on suspend
> > >
> > >
> > >
> > > [AMD Official Use Only]
> > >
> > >
> > >
> > > If that is done –
> > >
> > >
> > >
> > > + amdgpu_device_ip_set_powergating_state(adev,
> > AMD_IP_BLOCK_TYPE_UVD,
> > > + AMD_PG_STATE_GATE);
> > > + amdgpu_device_ip_set_clockgating_state(adev,
> > AMD_IP_BLOCK_TYPE_UVD,
> > > +
> > > + AMD_CG_STATE_GATE);
> > >
> > >
> > >
> > > Usual order is CG followed by PG. It comes in the else part, so less likely to
> > happen. Nice to fix for code correctness purpose.
> > >
> > > [Quan, Evan] Thanks Lijo. Make sense to me. However, actually these code
> > were copied from amdgpu_uvd_idle_work_handler() of amdgpu_uvd.c.
> > Same logic was used there. So, maybe @Zhu, James or @Liu, Leo can share
> > some insights about this.
> > >
> >
> > It looks like it is wrong there as well. We should be gating the clocks before
> > the power. The order is also wrong in amdgpu_uvd_ring_begin_use(). We
> > need to ungate the power before the clocks
> [Quan, Evan] I created a patch for this. But during the verification, I got the errors below
> [ 87.420822] [drm:uvd_v6_0_start [amdgpu]] *ERROR* UVD not responding, trying to reset the VCPU!!!
> [ 88.443029] [drm:uvd_v6_0_start [amdgpu]] *ERROR* UVD not responding, trying to reset the VCPU!!!
> [ 89.465386] [drm:uvd_v6_0_start [amdgpu]] *ERROR* UVD not responding, trying to reset the VCPU!!!
> [ 90.487629] [drm:uvd_v6_0_start [amdgpu]] *ERROR* UVD not responding, trying to reset the VCPU!!!
> [ 91.510380] [drm:uvd_v6_0_start [amdgpu]] *ERROR* UVD not responding, trying to reset the VCPU!!!
> [ 92.533782] [drm:uvd_v6_0_start [amdgpu]] *ERROR* UVD not responding, trying to reset the VCPU!!!
> [ 93.557400] [drm:uvd_v6_0_start [amdgpu]] *ERROR* UVD not responding, trying to reset the VCPU!!!
> [ 94.580708] [drm:uvd_v6_0_start [amdgpu]] *ERROR* UVD not responding, trying to reset the VCPU!!!
> [ 95.603832] [drm:uvd_v6_0_start [amdgpu]] *ERROR* UVD not responding, trying to reset the VCPU!!!
> [ 96.627727] [drm:uvd_v6_0_start [amdgpu]] *ERROR* UVD not responding, trying to reset the VCPU!!!
> [ 96.657453] [drm:uvd_v6_0_start [amdgpu]] *ERROR* UVD not responding, giving up!!!
> [ 96.665892] [drm:amdgpu_device_ip_set_powergating_state [amdgpu]] *ERROR* set_powergating_state of IP block <uvd_v6_0> failed -1
> [ 97.697422] amdgpu 0000:02:00.0: [drm:amdgpu_ib_ring_tests [amdgpu]] *ERROR* IB test failed on uvd (-110).
> [ 98.721432] amdgpu 0000:02:00.0: [drm:amdgpu_ib_ring_tests [amdgpu]] *ERROR* IB test failed on uvd_enc0 (-110).
> [ 99.745407] amdgpu 0000:02:00.0: [drm:amdgpu_ib_ring_tests [amdgpu]] *ERROR* IB test failed on uvd_enc1 (-110).
> [ 99.857784] [drm:amdgpu_device_delayed_init_work_handler [amdgpu]] *ERROR* ib ring test failed (-110).
>
> After checking the related source code roughly. It seems the underlaying implementation of -> set_powergating_state(e.g. uvd_v6_0_set_powergating_state ) performs more jobs than just power gating. And I guess maybe some of those jobs needs to be performed after -> set_clockgating_state. James and Leo may comment more.
>
Ok. Thanks for checking. I have no objections to the patch as is.
We can address the ordering questions later.
Alex
> BR
> Evan
> >
> > Alex
> >
> >
> > >
> > >
> > > BR
> > >
> > > Evan
> > >
> > >
> > >
> > > Thanks,
> > >
> > > Lijo
> > >
> > >
> > >
> > > From: Zhu, James <James.Zhu@xxxxxxx>
> > > Sent: Thursday, August 19, 2021 7:49 PM
> > > To: Quan, Evan <Evan.Quan@xxxxxxx>; amd-gfx@xxxxxxxxxxxxxxxxxxxxx
> > > Cc: Deucher, Alexander <Alexander.Deucher@xxxxxxx>; Chen, Guchun
> > > <Guchun.Chen@xxxxxxx>; Lazar, Lijo <Lijo.Lazar@xxxxxxx>; Pan, Xinhui
> > > <Xinhui.Pan@xxxxxxx>
> > > Subject: Re: [PATCH] drm/amdgpu: add missing cleanups for Polaris12
> > > UVD/VCE on suspend
> > >
> > >
> > >
> > > [AMD Official Use Only]
> > >
> > >
> > >
> > >
> > >
> > > Why not move changes into hw_fini?
> > >
> > >
> > >
> > > Best Regards!
> > >
> > >
> > >
> > > James Zhu
> > >
> > > ________________________________
> > >
> > > From: amd-gfx <amd-gfx-bounces@xxxxxxxxxxxxxxxxxxxxx> on behalf of
> > > Evan Quan <evan.quan@xxxxxxx>
> > > Sent: Wednesday, August 18, 2021 11:08 PM
> > > To: amd-gfx@xxxxxxxxxxxxxxxxxxxxx <amd-gfx@xxxxxxxxxxxxxxxxxxxxx>
> > > Cc: Deucher, Alexander <Alexander.Deucher@xxxxxxx>; Chen, Guchun
> > > <Guchun.Chen@xxxxxxx>; Lazar, Lijo <Lijo.Lazar@xxxxxxx>; Quan, Evan
> > > <Evan.Quan@xxxxxxx>; Pan, Xinhui <Xinhui.Pan@xxxxxxx>
> > > Subject: [PATCH] drm/amdgpu: add missing cleanups for Polaris12
> > > UVD/VCE on suspend
> > >
> > >
> > >
> > > Perform proper cleanups on UVD/VCE suspend: powergate enablement,
> > > clockgating enablement and dpm disablement. This can fix some hangs
> > > observed on suspending when UVD/VCE still using(e.g. issue
> > > "pm-suspend" when video is still playing).
> > >
> > > Change-Id: I36f39d9731e0a9638b52d5d92558b0ee9c23a9ed
> > > Signed-off-by: Evan Quan <evan.quan@xxxxxxx>
> > > Signed-off-by: xinhui pan <xinhui.pan@xxxxxxx>
> > > ---
> > > drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c | 24
> > ++++++++++++++++++++++++
> > > drivers/gpu/drm/amd/amdgpu/vce_v3_0.c | 23
> > +++++++++++++++++++++++
> > > 2 files changed, 47 insertions(+)
> > >
> > > diff --git a/drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c
> > > b/drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c
> > > index 4eebf973a065..d0fc6ec18c29 100644
> > > --- a/drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c
> > > +++ b/drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c
> > > @@ -554,6 +554,30 @@ static int uvd_v6_0_suspend(void *handle)
> > > int r;
> > > struct amdgpu_device *adev = (struct amdgpu_device *)handle;
> > >
> > > + /*
> > > + * Proper cleanups before halting the HW engine:
> > > + * - cancel the delayed idle work
> > > + * - enable powergating
> > > + * - enable clockgating
> > > + * - disable dpm
> > > + *
> > > + * TODO: to align with the VCN implementation, move the
> > > + * jobs for clockgating/powergating/dpm setting to
> > > + * ->set_powergating_state().
> > > + */
> > > + cancel_delayed_work_sync(&adev->uvd.idle_work);
> > > +
> > > + if (adev->pm.dpm_enabled) {
> > > + amdgpu_dpm_enable_uvd(adev, false);
> > > + } else {
> > > + amdgpu_asic_set_uvd_clocks(adev, 0, 0);
> > > + /* shutdown the UVD block */
> > > + amdgpu_device_ip_set_powergating_state(adev,
> > AMD_IP_BLOCK_TYPE_UVD,
> > > + AMD_PG_STATE_GATE);
> > > + amdgpu_device_ip_set_clockgating_state(adev,
> > AMD_IP_BLOCK_TYPE_UVD,
> > > + AMD_CG_STATE_GATE);
> > > + }
> > > +
> > > r = uvd_v6_0_hw_fini(adev);
> > > if (r)
> > > return r;
> > > diff --git a/drivers/gpu/drm/amd/amdgpu/vce_v3_0.c
> > > b/drivers/gpu/drm/amd/amdgpu/vce_v3_0.c
> > > index 6d9108fa22e0..a594ade5d30a 100644
> > > --- a/drivers/gpu/drm/amd/amdgpu/vce_v3_0.c
> > > +++ b/drivers/gpu/drm/amd/amdgpu/vce_v3_0.c
> > > @@ -503,6 +503,29 @@ static int vce_v3_0_suspend(void *handle)
> > > int r;
> > > struct amdgpu_device *adev = (struct amdgpu_device *)handle;
> > >
> > > + /*
> > > + * Proper cleanups before halting the HW engine:
> > > + * - cancel the delayed idle work
> > > + * - enable powergating
> > > + * - enable clockgating
> > > + * - disable dpm
> > > + *
> > > + * TODO: to align with the VCN implementation, move the
> > > + * jobs for clockgating/powergating/dpm setting to
> > > + * ->set_powergating_state().
> > > + */
> > > + cancel_delayed_work_sync(&adev->vce.idle_work);
> > > +
> > > + if (adev->pm.dpm_enabled) {
> > > + amdgpu_dpm_enable_vce(adev, false);
> > > + } else {
> > > + amdgpu_asic_set_vce_clocks(adev, 0, 0);
> > > + amdgpu_device_ip_set_powergating_state(adev,
> > AMD_IP_BLOCK_TYPE_VCE,
> > > + AMD_PG_STATE_GATE);
> > > + amdgpu_device_ip_set_clockgating_state(adev,
> > AMD_IP_BLOCK_TYPE_VCE,
> > > + AMD_CG_STATE_GATE);
> > > + }
> > > +
> > > r = vce_v3_0_hw_fini(adev);
> > > if (r)
> > > return r;
> > > --
> > > 2.29.0
