RE: [PATCH v3] drm/amdkfd: Fix partial migrate issue

["Deng, Emily" <Emily.Deng@xxxxxxx>]

[AMD Official Use Only - AMD Internal Distribution Only]

 

 

From: Yang, Philip <Philip.Yang@xxxxxxx>
Sent: Thursday, January 9, 2025 6:05 AM
To: Deng, Emily <Emily.Deng@xxxxxxx>; amd-gfx@xxxxxxxxxxxxxxxxxxxxx
Subject: Re: [PATCH v3] drm/amdkfd: Fix partial migrate issue

 

 

On 2025-01-08 08:19, Emily Deng wrote:

For partial migrate from ram to vram, the migrate->cpages is not

equal to migrate->npages, should use migrate->npages to check all needed

migrate pages which could be copied or not.

And only need to set those pages could be migrated to migrate->dst[i], or

the migrate_vma_pages will migrate the wrong pages based on the migrate->dst[i].

v2:

Add mpages to break the loop earlier.

v3:

Uses MIGRATE_PFN_MIGRATE to identify whether page could be migrated.

Signed-off-by: Emily Deng <Emily.Deng@xxxxxxx>

---

 drivers/gpu/drm/amd/amdkfd/kfd_migrate.c | 14 +++++++++-----

 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c

index eacfeb32f35d..b671d12c32f6 100644

--- a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c

+++ b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c

@@ -278,10 +278,11 @@ svm_migrate_copy_to_vram(struct kfd_node *node, struct svm_range *prange,

                         struct migrate_vma *migrate, struct dma_fence **mfence,

                         dma_addr_t *scratch, uint64_t ttm_res_offset)

 {

-       uint64_t npages = migrate->cpages;

+       uint64_t npages = migrate->npages;

        struct amdgpu_device *adev = node->adev;

        struct device *dev = adev->dev;

        struct amdgpu_res_cursor cursor;

+       uint64_t mpages = 0;

        dma_addr_t *src;

        uint64_t *dst;

        uint64_t i, j;

@@ -295,14 +296,17 @@ svm_migrate_copy_to_vram(struct kfd_node *node, struct svm_range *prange,

        amdgpu_res_first(prange->ttm_res, ttm_res_offset,

                         npages << PAGE_SHIFT, &cursor);

-       for (i = j = 0; i < npages; i++) {

+       for (i = j = 0; (i < npages) && (mpages < migrate->cpages); i++) {

                struct page *spage;

                dst[i] = cursor.start + (j << PAGE_SHIFT);

-               migrate->dst[i] = svm_migrate_addr_to_pfn(adev, dst[i]);

-        svm_migrate_get_vram_page(prange, migrate->dst[i]);

-               migrate->dst[i] = migrate_pfn(migrate->dst[i]);

+               if (migrate->src[i] & MIGRATE_PFN_MIGRATE) {

This should check flag MIGRATE_PFN_VALID, same as migrate_pfn_to_page, so move this inside below if (spage....)

[Emily]For the set_attrib for the first time for vram, the ram is invalid, but the MIGRATE_PFN_MIGRATE will be set.

 

Emily Deng

Best Wishes

 

 

+                       migrate->dst[i] = svm_migrate_addr_to_pfn(adev, dst[i]);

+                svm_migrate_get_vram_page(prange, migrate->dst[i]);

+                       migrate->dst[i] = migrate_pfn(migrate->dst[i]);

+                       mpages++;

increase mpages here is good, simpler than increasing mpages after each copy, as the last sdma copy outside loop will handle the remaining pages.

+               }

                spage = migrate_pfn_to_page(migrate->src[i]);

                if (spage && !is_zone_device_page(spage)) {

                        src[i] = dma_map_page(dev, spage, 0, PAGE_SIZE,

out_free_vram_pages path error handling need fix to put, unlock all migrated pages too, please check changes attached, I only compile but didn't run test with the change.

Regards,

Philip