Re: [PATCH] drm/ttm: specify bo priority when initializing ttm bo

["Zhang, Jerry (Junwei)" <Jerry.Zhang@xxxxxxx>]

On 05/11/2018 10:26 AM, zhoucm1 wrote:
> On 2018年05月11日 10:19, Zhang, Jerry (Junwei) wrote:
> > On 05/11/2018 10:11 AM, zhoucm1 wrote:
> > > On 2018年05月11日 09:21, Zhang, Jerry (Junwei) wrote:
> > > > On 05/10/2018 10:40 PM, Christian König wrote:
> > > > > Am 10.05.2018 um 07:01 schrieb Junwei Zhang:
> > > > > > Expect to add an evitable bo who has reservation object
> > > > > > to the correct lru[bo->priority] list
> > > > >
> > > > > Nice catch, but since this affects only a very small use case can we just
> > > > > remove
> > > > > and readd the BO to the LRU?
> > > > >
> > > > > A call to ttm_bo_move_to_lru_tail() after setting the priority should be
> > > > > sufficient.
> > > >
> > > > Yeah, at that moment, I thought setting a priority for tbo at initialization
> > > > is a general way, which may help all associated modules to get out of
> > > > potential pain if more priorities are adopted.
> > > >
> > > > Agree.
> > > > The most simple way is to move the bo back to expected lru after priority
> > > > setting. Going to prepare a patch.
> > > No, there is more simpler way.  see inline...
> > >
> > > > (in this case, include amd-gfx mail only)
> > > >
> > > > Regards,
> > > > Jerry
> > > >
> > > > > Thanks,
> > > > > Christian.
> > > > >
> > > > > > Signed-off-by: Junwei Zhang <Jerry.Zhang@xxxxxxx>
> > > > > > ---
> > > > > >   drivers/gpu/drm/amd/amdgpu/amdgpu_object.c  | 11 ++++++-----
> > > > > >   drivers/gpu/drm/ast/ast_ttm.c               |  2 +-
> > > > > >   drivers/gpu/drm/bochs/bochs_mm.c            |  2 +-
> > > > > >   drivers/gpu/drm/cirrus/cirrus_ttm.c         |  2 +-
> > > > > >   drivers/gpu/drm/hisilicon/hibmc/hibmc_ttm.c |  2 +-
> > > > > >   drivers/gpu/drm/mgag200/mgag200_ttm.c       |  2 +-
> > > > > >   drivers/gpu/drm/nouveau/nouveau_bo.c        |  2 +-
> > > > > >   drivers/gpu/drm/qxl/qxl_object.c            |  2 +-
> > > > > >   drivers/gpu/drm/radeon/radeon_object.c      |  2 +-
> > > > > >   drivers/gpu/drm/ttm/ttm_bo.c                |  8 +++++---
> > > > > >   drivers/gpu/drm/virtio/virtgpu_object.c     |  2 +-
> > > > > >   drivers/gpu/drm/vmwgfx/vmwgfx_resource.c    |  2 +-
> > > > > >   drivers/staging/vboxvideo/vbox_ttm.c        |  2 +-
> > > > > >   include/drm/ttm/ttm_bo_api.h                |  5 ++++-
> > > > > >   14 files changed, 26 insertions(+), 20 deletions(-)
> > > > > >
> > > > > > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
> > > > > > b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
> > > > > > index e62153a..9a25ecb 100644
> > > > > > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
> > > > > > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
> > > > > > @@ -360,6 +360,7 @@ static int amdgpu_bo_do_create(struct amdgpu_device
> > > > > > *adev,
> > > > > >       };
> > > > > >       struct amdgpu_bo *bo;
> > > > > >       unsigned long page_align, size = bp->size;
> > > > > > +    uint32_t prio = 0;
> > > > > >       size_t acc_size;
> > > > > >       int r;
> > > > > > @@ -419,10 +420,13 @@ static int amdgpu_bo_do_create(struct amdgpu_device
> > > > > > *adev,
> > > > > >       bo->tbo.bdev = &adev->mman.bdev;
> > > > > >       amdgpu_ttm_placement_from_domain(bo, bp->domain);
> > > > > > +    if (bp->type == ttm_bo_type_kernel)
> > > > > > +        prio = 1;
> > > Why not just bo->tbo.priority = 1? In that case, you don't need to add more
> > > parameter to ttm_bo_init_reserved.
> > > In fact, this change is already in my per-vm-lru patch set.
> >
> > Do you mean specify 1 before ttm_bo_init_reserved()?
> >
> > ttm_bo_init_reserved() will overwrite it to 0 again inside.
> If it can set outside of ttm, you can remove that overwriting. tbo allocation is
> from kzalloc.

Yeah, that bases on different coding idea, I think.
1) always to clear the tbo memory before tbo init (mostly, we do so)
2) make sure the key(all) members are initialized as expected always inside init 
function.

anyway, I will sent it out firstly and RFC.

Jerry

> Regards,
> David Zhou
> > Jerry
> >
> > > Regards,
> > > David Zhou
> > > > > >       r = ttm_bo_init_reserved(&adev->mman.bdev, &bo->tbo, size, bp->type,
> > > > > > -                 &bo->placement, page_align, &ctx, acc_size,
> > > > > > -                 NULL, bp->resv, &amdgpu_ttm_bo_destroy);
> > > > > > +                 prio, &bo->placement, page_align, &ctx,
> > > > > > +                 acc_size, NULL, bp->resv,
> > > > > > +                 &amdgpu_ttm_bo_destroy);
> > > > > >       if (unlikely(r != 0))
> > > > > >           return r;
> > > > > > @@ -434,9 +438,6 @@ static int amdgpu_bo_do_create(struct amdgpu_device
> > > > > > *adev,
> > > > > >       else
> > > > > >           amdgpu_cs_report_moved_bytes(adev, ctx.bytes_moved, 0);
> > > > > > -    if (bp->type == ttm_bo_type_kernel)
> > > > > > -        bo->tbo.priority = 1;
> > > > > > -
> > > > > >       if (bp->flags & AMDGPU_GEM_CREATE_VRAM_CLEARED &&
> > > > > >           bo->tbo.mem.placement & TTM_PL_FLAG_VRAM) {
> > > > > >           struct dma_fence *fence;
> > > > > > diff --git a/drivers/gpu/drm/ast/ast_ttm.c b/drivers/gpu/drm/ast/ast_ttm.c
> > > > > > index fe354eb..aabb96a 100644
> > > > > > --- a/drivers/gpu/drm/ast/ast_ttm.c
> > > > > > +++ b/drivers/gpu/drm/ast/ast_ttm.c
> > > > > > @@ -320,7 +320,7 @@ int ast_bo_create(struct drm_device *dev, int size, int
> > > > > > align,
> > > > > >                          sizeof(struct ast_bo));
> > > > > >       ret = ttm_bo_init(&ast->ttm.bdev, &astbo->bo, size,
> > > > > > -              ttm_bo_type_device, &astbo->placement,
> > > > > > +              ttm_bo_type_device, 0, &astbo->placement,
> > > > > >                 align >> PAGE_SHIFT, false, acc_size,
> > > > > >                 NULL, NULL, ast_bo_ttm_destroy);
> > > > > >       if (ret)
> > > > > > diff --git a/drivers/gpu/drm/bochs/bochs_mm.c
> > > > > > b/drivers/gpu/drm/bochs/bochs_mm.c
> > > > > > index 39cd084..9693109 100644
> > > > > > --- a/drivers/gpu/drm/bochs/bochs_mm.c
> > > > > > +++ b/drivers/gpu/drm/bochs/bochs_mm.c
> > > > > > @@ -366,7 +366,7 @@ static int bochs_bo_create(struct drm_device *dev, int
> > > > > > size, int align,
> > > > > >                          sizeof(struct bochs_bo));
> > > > > >       ret = ttm_bo_init(&bochs->ttm.bdev, &bochsbo->bo, size,
> > > > > > -              ttm_bo_type_device, &bochsbo->placement,
> > > > > > +              ttm_bo_type_device, 0, &bochsbo->placement,
> > > > > >                 align >> PAGE_SHIFT, false, acc_size,
> > > > > >                 NULL, NULL, bochs_bo_ttm_destroy);
> > > > > >       if (ret)
> > > > > > diff --git a/drivers/gpu/drm/cirrus/cirrus_ttm.c
> > > > > > b/drivers/gpu/drm/cirrus/cirrus_ttm.c
> > > > > > index f219532..c1d85f8 100644
> > > > > > --- a/drivers/gpu/drm/cirrus/cirrus_ttm.c
> > > > > > +++ b/drivers/gpu/drm/cirrus/cirrus_ttm.c
> > > > > > @@ -327,7 +327,7 @@ int cirrus_bo_create(struct drm_device *dev, int size,
> > > > > > int
> > > > > > align,
> > > > > >                          sizeof(struct cirrus_bo));
> > > > > >       ret = ttm_bo_init(&cirrus->ttm.bdev, &cirrusbo->bo, size,
> > > > > > -              ttm_bo_type_device, &cirrusbo->placement,
> > > > > > +              ttm_bo_type_device, 0, &cirrusbo->placement,
> > > > > >                 align >> PAGE_SHIFT, false, acc_size,
> > > > > >                 NULL, NULL, cirrus_bo_ttm_destroy);
> > > > > >       if (ret)
> > > > > > diff --git a/drivers/gpu/drm/hisilicon/hibmc/hibmc_ttm.c
> > > > > > b/drivers/gpu/drm/hisilicon/hibmc/hibmc_ttm.c
> > > > > > index 4871025..8c24731 100644
> > > > > > --- a/drivers/gpu/drm/hisilicon/hibmc/hibmc_ttm.c
> > > > > > +++ b/drivers/gpu/drm/hisilicon/hibmc/hibmc_ttm.c
> > > > > > @@ -315,7 +315,7 @@ int hibmc_bo_create(struct drm_device *dev, int size, int
> > > > > > align,
> > > > > >                          sizeof(struct hibmc_bo));
> > > > > >       ret = ttm_bo_init(&hibmc->bdev, &hibmcbo->bo, size,
> > > > > > -              ttm_bo_type_device, &hibmcbo->placement,
> > > > > > +              ttm_bo_type_device, 0, &hibmcbo->placement,
> > > > > >                 align >> PAGE_SHIFT, false, acc_size,
> > > > > >                 NULL, NULL, hibmc_bo_ttm_destroy);
> > > > > >       if (ret) {
> > > > > > diff --git a/drivers/gpu/drm/mgag200/mgag200_ttm.c
> > > > > > b/drivers/gpu/drm/mgag200/mgag200_ttm.c
> > > > > > index 05570f0..e4db046 100644
> > > > > > --- a/drivers/gpu/drm/mgag200/mgag200_ttm.c
> > > > > > +++ b/drivers/gpu/drm/mgag200/mgag200_ttm.c
> > > > > > @@ -323,7 +323,7 @@ int mgag200_bo_create(struct drm_device *dev, int size,
> > > > > > int align,
> > > > > >                          sizeof(struct mgag200_bo));
> > > > > >       ret = ttm_bo_init(&mdev->ttm.bdev, &mgabo->bo, size,
> > > > > > -              ttm_bo_type_device, &mgabo->placement,
> > > > > > +              ttm_bo_type_device, 0, &mgabo->placement,
> > > > > >                 align >> PAGE_SHIFT, false, acc_size,
> > > > > >                 NULL, NULL, mgag200_bo_ttm_destroy);
> > > > > >       if (ret)
> > > > > > diff --git a/drivers/gpu/drm/nouveau/nouveau_bo.c
> > > > > > b/drivers/gpu/drm/nouveau/nouveau_bo.c
> > > > > > index 6f402c4..8a6f949 100644
> > > > > > --- a/drivers/gpu/drm/nouveau/nouveau_bo.c
> > > > > > +++ b/drivers/gpu/drm/nouveau/nouveau_bo.c
> > > > > > @@ -297,7 +297,7 @@
> > > > > >                          sizeof(struct nouveau_bo));
> > > > > >       ret = ttm_bo_init(&drm->ttm.bdev, &nvbo->bo, size,
> > > > > > -              type, &nvbo->placement,
> > > > > > +              type, 0, &nvbo->placement,
> > > > > >                 align >> PAGE_SHIFT, false, acc_size, sg,
> > > > > >                 robj, nouveau_bo_del_ttm);
> > > > > >       if (ret) {
> > > > > > diff --git a/drivers/gpu/drm/qxl/qxl_object.c
> > > > > > b/drivers/gpu/drm/qxl/qxl_object.c
> > > > > > index 6a30196..3e8c64b 100644
> > > > > > --- a/drivers/gpu/drm/qxl/qxl_object.c
> > > > > > +++ b/drivers/gpu/drm/qxl/qxl_object.c
> > > > > > @@ -108,7 +108,7 @@ int qxl_bo_create(struct qxl_device *qdev,
> > > > > >       qxl_ttm_placement_from_domain(bo, domain, pinned);
> > > > > > -    r = ttm_bo_init(&qdev->mman.bdev, &bo->tbo, size, type,
> > > > > > +    r = ttm_bo_init(&qdev->mman.bdev, &bo->tbo, size, type, 0,
> > > > > >               &bo->placement, 0, !kernel, size,
> > > > > >               NULL, NULL, &qxl_ttm_bo_destroy);
> > > > > >       if (unlikely(r != 0)) {
> > > > > > diff --git a/drivers/gpu/drm/radeon/radeon_object.c
> > > > > > b/drivers/gpu/drm/radeon/radeon_object.c
> > > > > > index edbb4cd..b274e3f 100644
> > > > > > --- a/drivers/gpu/drm/radeon/radeon_object.c
> > > > > > +++ b/drivers/gpu/drm/radeon/radeon_object.c
> > > > > > @@ -256,7 +256,7 @@ int radeon_bo_create(struct radeon_device *rdev,
> > > > > >       radeon_ttm_placement_from_domain(bo, domain);
> > > > > >       /* Kernel allocation are uninterruptible */
> > > > > >       down_read(&rdev->pm.mclk_lock);
> > > > > > -    r = ttm_bo_init(&rdev->mman.bdev, &bo->tbo, size, type,
> > > > > > +    r = ttm_bo_init(&rdev->mman.bdev, &bo->tbo, size, type, 0,
> > > > > >               &bo->placement, page_align, !kernel, acc_size,
> > > > > >               sg, resv, &radeon_ttm_bo_destroy);
> > > > > >       up_read(&rdev->pm.mclk_lock);
> > > > > > diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c
> > > > > > index 98e06f8..d1f0585 100644
> > > > > > --- a/drivers/gpu/drm/ttm/ttm_bo.c
> > > > > > +++ b/drivers/gpu/drm/ttm/ttm_bo.c
> > > > > > @@ -1109,6 +1109,7 @@ int ttm_bo_init_reserved(struct ttm_bo_device *bdev,
> > > > > >                struct ttm_buffer_object *bo,
> > > > > >                unsigned long size,
> > > > > >                enum ttm_bo_type type,
> > > > > > +             uint32_t priority,
> > > > > >                struct ttm_placement *placement,
> > > > > >                uint32_t page_alignment,
> > > > > >                struct ttm_operation_ctx *ctx,
> > > > > > @@ -1175,7 +1176,7 @@ int ttm_bo_init_reserved(struct ttm_bo_device *bdev,
> > > > > >       reservation_object_init(&bo->ttm_resv);
> > > > > >       atomic_inc(&bo->bdev->glob->bo_count);
> > > > > >       drm_vma_node_reset(&bo->vma_node);
> > > > > > -    bo->priority = 0;
> > > > > > +    bo->priority = priority;
> > > > > >       /*
> > > > > >        * For ttm_bo_type_device buffers, allocate
> > > > > > @@ -1219,6 +1220,7 @@ int ttm_bo_init(struct ttm_bo_device *bdev,
> > > > > >           struct ttm_buffer_object *bo,
> > > > > >           unsigned long size,
> > > > > >           enum ttm_bo_type type,
> > > > > > +        uint32_t priority,
> > > > > >           struct ttm_placement *placement,
> > > > > >           uint32_t page_alignment,
> > > > > >           bool interruptible,
> > > > > > @@ -1230,7 +1232,7 @@ int ttm_bo_init(struct ttm_bo_device *bdev,
> > > > > >       struct ttm_operation_ctx ctx = { interruptible, false };
> > > > > >       int ret;
> > > > > > -    ret = ttm_bo_init_reserved(bdev, bo, size, type, placement,
> > > > > > +    ret = ttm_bo_init_reserved(bdev, bo, size, type, priority, placement,
> > > > > >                      page_alignment, &ctx, acc_size,
> > > > > >                      sg, resv, destroy);
> > > > > >       if (ret)
> > > > > > @@ -1288,7 +1290,7 @@ int ttm_bo_create(struct ttm_bo_device *bdev,
> > > > > >           return -ENOMEM;
> > > > > >       acc_size = ttm_bo_acc_size(bdev, size, sizeof(struct
> > > > > > ttm_buffer_object));
> > > > > > -    ret = ttm_bo_init(bdev, bo, size, type, placement, page_alignment,
> > > > > > +    ret = ttm_bo_init(bdev, bo, size, type, 0, placement, page_alignment,
> > > > > >                 interruptible, acc_size,
> > > > > >                 NULL, NULL, NULL);
> > > > > >       if (likely(ret == 0))
> > > > > > diff --git a/drivers/gpu/drm/virtio/virtgpu_object.c
> > > > > > b/drivers/gpu/drm/virtio/virtgpu_object.c
> > > > > > index 9f2f470..c11eba8 100644
> > > > > > --- a/drivers/gpu/drm/virtio/virtgpu_object.c
> > > > > > +++ b/drivers/gpu/drm/virtio/virtgpu_object.c
> > > > > > @@ -88,7 +88,7 @@ int virtio_gpu_object_create(struct virtio_gpu_device
> > > > > > *vgdev,
> > > > > >       bo->dumb = false;
> > > > > >       virtio_gpu_init_ttm_placement(bo, pinned);
> > > > > > -    ret = ttm_bo_init(&vgdev->mman.bdev, &bo->tbo, size, type,
> > > > > > +    ret = ttm_bo_init(&vgdev->mman.bdev, &bo->tbo, size, type, 0,
> > > > > >                 &bo->placement, 0, !kernel, acc_size,
> > > > > >                 NULL, NULL, &virtio_gpu_ttm_bo_destroy);
> > > > > >       /* ttm_bo_init failure will call the destroy */
> > > > > > diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_resource.c
> > > > > > b/drivers/gpu/drm/vmwgfx/vmwgfx_resource.c
> > > > > > index 6b3a942..e7a4216 100644
> > > > > > --- a/drivers/gpu/drm/vmwgfx/vmwgfx_resource.c
> > > > > > +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_resource.c
> > > > > > @@ -385,7 +385,7 @@ int vmw_dmabuf_init(struct vmw_private *dev_priv,
> > > > > >       INIT_LIST_HEAD(&vmw_bo->res_list);
> > > > > >       ret = ttm_bo_init(bdev, &vmw_bo->base, size,
> > > > > > -              ttm_bo_type_device, placement,
> > > > > > +              ttm_bo_type_device, 0, placement,
> > > > > >                 0, interruptible, acc_size,
> > > > > >                 NULL, NULL, bo_free);
> > > > > >       return ret;
> > > > > > diff --git a/drivers/staging/vboxvideo/vbox_ttm.c
> > > > > > b/drivers/staging/vboxvideo/vbox_ttm.c
> > > > > > index 548edb7..b93f8f0 100644
> > > > > > --- a/drivers/staging/vboxvideo/vbox_ttm.c
> > > > > > +++ b/drivers/staging/vboxvideo/vbox_ttm.c
> > > > > > @@ -329,7 +329,7 @@ int vbox_bo_create(struct drm_device *dev, int size, int
> > > > > > align,
> > > > > >                          sizeof(struct vbox_bo));
> > > > > >       ret = ttm_bo_init(&vbox->ttm.bdev, &vboxbo->bo, size,
> > > > > > -              ttm_bo_type_device, &vboxbo->placement,
> > > > > > +              ttm_bo_type_device, 0, &vboxbo->placement,
> > > > > >                 align >> PAGE_SHIFT, false, acc_size,
> > > > > >                 NULL, NULL, vbox_bo_ttm_destroy);
> > > > > >       if (ret)
> > > > > > diff --git a/include/drm/ttm/ttm_bo_api.h b/include/drm/ttm/ttm_bo_api.h
> > > > > > index c67977a..f4142af 100644
> > > > > > --- a/include/drm/ttm/ttm_bo_api.h
> > > > > > +++ b/include/drm/ttm/ttm_bo_api.h
> > > > > > @@ -464,6 +464,7 @@ size_t ttm_bo_dma_acc_size(struct ttm_bo_device *bdev,
> > > > > >    * @bo: Pointer to a ttm_buffer_object to be initialized.
> > > > > >    * @size: Requested size of buffer object.
> > > > > >    * @type: Requested type of buffer object.
> > > > > > + * @priority: Requested priority of buffer object.
> > > > > >    * @flags: Initial placement flags.
> > > > > >    * @page_alignment: Data alignment in pages.
> > > > > >    * @ctx: TTM operation context for memory allocation.
> > > > > > @@ -496,6 +497,7 @@ int ttm_bo_init_reserved(struct ttm_bo_device *bdev,
> > > > > >                struct ttm_buffer_object *bo,
> > > > > >                unsigned long size,
> > > > > >                enum ttm_bo_type type,
> > > > > > +             uint32_t priority,
> > > > > >                struct ttm_placement *placement,
> > > > > >                uint32_t page_alignment,
> > > > > >                struct ttm_operation_ctx *ctx,
> > > > > > @@ -511,6 +513,7 @@ int ttm_bo_init_reserved(struct ttm_bo_device *bdev,
> > > > > >    * @bo: Pointer to a ttm_buffer_object to be initialized.
> > > > > >    * @size: Requested size of buffer object.
> > > > > >    * @type: Requested type of buffer object.
> > > > > > + * @priority: Requested priority of buffer object.
> > > > > >    * @flags: Initial placement flags.
> > > > > >    * @page_alignment: Data alignment in pages.
> > > > > >    * @interruptible: If needing to sleep to wait for GPU resources,
> > > > > > @@ -542,7 +545,7 @@ int ttm_bo_init_reserved(struct ttm_bo_device *bdev,
> > > > > >    * -ERESTARTSYS: Interrupted by signal while sleeping waiting for
> > > > > > resources.
> > > > > >    */
> > > > > >   int ttm_bo_init(struct ttm_bo_device *bdev, struct ttm_buffer_object *bo,
> > > > > > -        unsigned long size, enum ttm_bo_type type,
> > > > > > +        unsigned long size, enum ttm_bo_type type, uint32_t priority,
> > > > > >           struct ttm_placement *placement,
> > > > > >           uint32_t page_alignment, bool interrubtible, size_t acc_size,
> > > > > >           struct sg_table *sg, struct reservation_object *resv,
> > > > _______________________________________________
> > > > amd-gfx mailing list
> > > > amd-gfx@xxxxxxxxxxxxxxxxxxxxx
> > > > https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
dri-devel mailing list
dri-devel@xxxxxxxxxxxxxxxxxxxxx
https://lists.freedesktop.org/mailman/listinfo/dri-devel