Am 13.12.2017 um 04:42 schrieb Monk Liu:
> a new gart flush tlb function implemented for SRIOV,
> and invoke it during RUNTIME for gart flush TLBs
>
> this could avoid the issue that gart flush (via CPU MMIO)
> being interrupted by word switch which lead to DMAR error
> on Host/IOMMU side, with this function the gart flush
> tlbs always run on KIQ with single PM4 package so it won't
> get interrupted before the flushing finished.
>
> Change-Id: I0849658d7945c3874b3cc0d9369a50e1aedb8312
> Signed-off-by: Monk Liu <Monk.Liu at amd.com>
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c | 27 +++++++++++++++++++++++++++
> drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h | 1 +
> drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c | 3 +++
> 3 files changed, 31 insertions(+)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
> index e7dfb7b..7a6ef64 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
> @@ -172,6 +172,33 @@ void amdgpu_virt_kiq_wreg(struct amdgpu_device *adev, uint32_t reg, uint32_t v)
> DRM_ERROR("wait for kiq fence error: %ld\n", r);
> }
>
> +int amdgpu_virt_gart_flush_tlbs(struct amdgpu_device *adev)
> +{
> + struct amdgpu_kiq *kiq = &adev->gfx.kiq;
> + struct amdgpu_ring *ring = &kiq->ring;
> + unsigned long flags;
> + signed long r;
> + uint32_t seq;
> +
> + if(!ring->funcs->emit_invalidate_tlbs)
> + return -ENOENT;
> +
> + spin_lock_irqsave(&kiq->ring_lock, flags);
> + amdgpu_ring_alloc(ring, 16);
> + amdgpu_ring_emit_invalidate_tlbs(ring);
> + amdgpu_fence_emit_polling(ring, &seq);
> + amdgpu_ring_commit(ring);
> + spin_unlock_irqrestore(&kiq->ring_lock, flags);
> +
> + r = amdgpu_fence_wait_polling(ring, seq, MAX_KIQ_REG_WAIT);
> + if (r < 1) {
> + DRM_ERROR("wait for kiq invalidate tlbs error: %ld\n", r);
> + return -ETIME;
> + }
> +
> + return 0;
> +}
> +
> /**
> * amdgpu_virt_request_full_gpu() - request full gpu access
> * @amdgpu: amdgpu device.
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h
> index 6a83425..935fed3 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h
> @@ -297,5 +297,6 @@ int amdgpu_virt_fw_reserve_get_checksum(void *obj, unsigned long obj_size,
> unsigned int key,
> unsigned int chksum);
> void amdgpu_virt_init_data_exchange(struct amdgpu_device *adev);
> +int amdgpu_virt_gart_flush_tlbs(struct amdgpu_device *adev);
>
> #endif
> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
> index 1b5dfcc..a195039 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
> @@ -332,6 +332,9 @@ static void gmc_v9_0_gart_flush_gpu_tlb(struct amdgpu_device *adev,
> /* flush hdp cache */
> adev->nbio_funcs->hdp_flush(adev);
>
> + if (amdgpu_sriov_runtime(adev) && !amdgpu_virt_gart_flush_tlbs(adev))
> + return;
> +
Better open code that like this:
if (amdgpu_sriov_runtime(adev)) {
   /* Try using the KIQ */
   r = amdgpu_virt_gart_flush_tlbs(adev));
   if (!r)
      return;
}
Apart from that it looks good to me.
Christian.
> spin_lock(&adev->mc.invalidate_lock);
>
> for (i = 0; i < AMDGPU_MAX_VMHUBS; ++i) {
