Re: [PATCH 28/29] drm/amdkfd: Refactor migrate init to support partition switch

[Philip Yang <yangp@xxxxxxx>]

On 2023-07-21 04:55, Michel Dänzer wrote:

> On 7/20/23 22:48, Philip Yang wrote:
> > On 2023-07-20 06:46, Michel Dänzer wrote:
> > > On 7/17/23 15:09, Michel Dänzer wrote:
> > > > On 5/10/23 23:23, Alex Deucher wrote:
> > > > > From: Philip Yang <Philip.Yang@xxxxxxx>
> > > > >
> > > > > Rename smv_migrate_init to a better name kgd2kfd_init_zone_device
> > > > > because it setup zone devive pgmap for page migration and keep it in
> > > > > kfd_migrate.c to access static functions svm_migrate_pgmap_ops. Call it
> > > > > only once in amdgpu_device_ip_init after adev ip blocks are initialized,
> > > > > but before amdgpu_amdkfd_device_init initialize kfd nodes which enable
> > > > > SVM support based on pgmap.
> > > > >
> > > > > svm_range_set_max_pages is called by kgd2kfd_device_init everytime after
> > > > > switching compute partition mode.
> > > > >
> > > > > Signed-off-by: Philip Yang <Philip.Yang@xxxxxxx>
> > > > > Reviewed-by: Felix Kuehling <Felix.Kuehling@xxxxxxx>
> > > > > Signed-off-by: Alex Deucher <alexander.deucher@xxxxxxx>
> > > > >             
> > > > I bisected a regression to this commit, which broke HW acceleration on this ThinkPad E595 with Picasso APU.
> > > >           
> > > Actually, it doesn't seem to break HW acceleration completely. GDM eventually comes up with HW acceleration, it takes a long time (~30s or so) to start up though.
> > >
> > > Later, the same messages as described in https://gitlab.freedesktop.org/drm/amd/-/issues/2659 appear.
> > >
> > > Reverting this commit fixes all of the above symptoms.
> > >
> > >
> > > I reproduced all of the above symptoms with amd-staging-drm-next commit 75515acf4b60 ("i2c: nvidia-gpu: Add ACPI property to align with device-tree") as well.
> > >
> > >
> > > For full disclosure, I use these kernel command line arguments:
> > >
> > >  fbcon=font:10x18 drm_kms_helper.drm_fbdev_overalloc=112 amdgpu.noretry=1 amdgpu.mcbp=1
> > >         
> > Thanks for the issue report and full disclosure, but I am not able to reproduce this issue, with both drm-next branch and amd-staging-drm-next branch tip on gitlab. The test system has same device id, running Ubuntu 22.04, latest linux-firmware-20230625.tar.gz, and same BIOS version.
> >       
> FWIW, your system has PCI revision ID 0xC2, while mine has 0xC1.
>
> Also, I'm currently using linux-firmware 20230515. AFAICT there are no relevant changes in 20230625, but I'm attaching the contents of /sys/kernel/debug/dri/0/amdgpu_firmware_info just in case.
>
>
> > I attached full dmesg log, could you help check if there is other difference, maybe kernel config, gcc version... it is hard to guess what could cause the basic driver gfx ring IB test timeout.
> >       
> I suspect the IOMMU page faults logged in my dmesg might be relevant:
>
>  amdgpu: Topology: Add APU node [0x15d8:0x1002]
>  amdgpu 0000:05:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0000 address=0x122201800 flags=0x0070]
>  amdgpu 0000:05:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0000 address=0x1125fe380 flags=0x0070]
>  kfd kfd: amdgpu: added device 1002:15d8
>
> There are no such page faults with the commit reverted.
>
> Other than that and the IB test failure messages, our dmesg outputs are mostly identical indeed.
>

Yes, I don't have IO_PAGE_FAULT message on my system, thanks for the finding, I will continue investigating the root cause.

You are right, the error message could cause gfx ring IB test timeout failure, this patch does change the order of driver memory allocation. IOMMU is in translation mode on Ubuntu config.

To help confirm if this is caused by IOMMU, please add this to kernel boot option to set IOMMU to passthrough mode, check if this can workaround the issue

iommu=pt

Regards,

Philip

>