|
[AMD Official Use Only - Internal Distribution Only]
Ok, thanks very much Alex!
From: Alex Deucher <alexdeucher@xxxxxxxxx>
Sent: Thursday, January 9, 2020 11:12 To: Yin, Tianci (Rico) <Tianci.Yin@xxxxxxx> Cc: Christian König <ckoenig.leichtzumerken@xxxxxxxxx>; Koenig, Christian <Christian.Koenig@xxxxxxx>; Long, Gang <Gang.Long@xxxxxxx>; Wang, Kevin(Yang) <Kevin1.Wang@xxxxxxx>; Xu, Feifei <Feifei.Xu@xxxxxxx>; amd-gfx list <amd-gfx@xxxxxxxxxxxxxxxxxxxxx>; Tuikov, Luben <Luben.Tuikov@xxxxxxx>; Deucher, Alexander <Alexander.Deucher@xxxxxxx>; Yuan, Xiaojie <Xiaojie.Yuan@xxxxxxx>; Zhang, Hawking <Hawking.Zhang@xxxxxxx> Subject: Re: [PATCH] drm/amdgpu: fix modprobe failure of the 2nd GPU when GDDR6 training enabled On Wed, Jan 8, 2020 at 10:07 PM Yin, Tianci (Rico) <Tianci.Yin@xxxxxxx> wrote:
> > [AMD Official Use Only - Internal Distribution Only] > > > Thanks Alex and Christian! > > Hi Christian, > > On ASICs with gmc v10, I find amdgpu_bo_late_init() is not invoked, so stolen memory never get free, on other ASICs with gmc v9/v8/v7/v6, stolen memory was freed in gmc_v*_0_late_init(). I don't know why, are there some special reasons or just missed by coding? > Looks like it should be added. Possibly got lost when we merged the navi code from the topic branch. Alex > Thanks! > > Rico > > ________________________________ > From: Christian König <ckoenig.leichtzumerken@xxxxxxxxx> > Sent: Wednesday, January 8, 2020 23:04 > To: Alex Deucher <alexdeucher@xxxxxxxxx>; Koenig, Christian <Christian.Koenig@xxxxxxx> > Cc: Long, Gang <Gang.Long@xxxxxxx>; Wang, Kevin(Yang) <Kevin1.Wang@xxxxxxx>; Xu, Feifei <Feifei.Xu@xxxxxxx>; Yin, Tianci (Rico) <Tianci.Yin@xxxxxxx>; amd-gfx list <amd-gfx@xxxxxxxxxxxxxxxxxxxxx>; Tuikov, Luben <Luben.Tuikov@xxxxxxx>; Deucher, Alexander <Alexander.Deucher@xxxxxxx>; Yuan, Xiaojie <Xiaojie.Yuan@xxxxxxx>; Zhang, Hawking <Hawking.Zhang@xxxxxxx> > Subject: Re: [PATCH] drm/amdgpu: fix modprobe failure of the 2nd GPU when GDDR6 training enabled > > Am 08.01.20 um 15:49 schrieb Alex Deucher: > > On Wed, Jan 8, 2020 at 7:52 AM Christian König <christian.koenig@xxxxxxx> wrote: > >> Am 08.01.20 um 13:36 schrieb Tianci Yin: > >>> From: "Tianci.Yin" <tianci.yin@xxxxxxx> > >>> > >>> [why] > >>> In dual GPUs scenario, stolen_size is assigned to zero on the 2nd GPU, > >>> then the bottom region of VRAM was allocated as GTT, unfortunately > >>> a small region of bottom VRAM was encroached by UMC firmware during > >>> GDDR6 BIST training, this cause pagefault. > >> What I'm missing here is why is the stolen size zero on the 2nd GPU? > >> > >> Maybe we need to read the stolen size after posting the GPU instead? > > There is no stolen memory on secondary GPUs because there is no pre-OS > > console using that memory. > > Ah! Yeah that makes sense. > > But in this case I would say we should change the patch like the following: > > adev->gmc.stolen_size = min(adev->gmc.stolen_size, > AMDGPU_STOLEN_VGA_DEFAULT_SIZE); > > And in amdgpu_ttm_late_init(): > > /* Can't free the stolen VGA memory when it might be used for memory > training again. */ > if (!adev->fw_vram_usage.mem_train_support) > amdgpu_bo_free_kernel(.... > > > Regards, > Christian. > > > > > > Alex > > > >> Regards, > >> Christian. > >> > >>> [how] > >>> Forcing stolen_size to 3MB, then the bottom region of VRAM was > >>> allocated as stolen memory, GTT corruption avoid. > >>> The stolen memory of the 2nd GPU will be free in late_init phase, > >>> no memory wasted. > >>> > >>> Change-Id: Icd0ad7de41333282949bb1e3e676c6c307ddd081 > >>> Signed-off-by: Tianci.Yin <tianci.yin@xxxxxxx> > >>> --- > >>> drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h | 6 ++++++ > >>> drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c | 21 +++++++++++++++++++++ > >>> 2 files changed, 27 insertions(+) > >>> > >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h > >>> index c91dd602d5f1..440b793316df 100644 > >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h > >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h > >>> @@ -60,6 +60,11 @@ > >>> */ > >>> #define AMDGPU_GMC_FAULT_TIMEOUT 5000ULL > >>> > >>> +/* > >>> + * Default stolen memory size, 1024 * 768 * 4 > >>> + */ > >>> +#define AMDGPU_STOLEN_VGA_DEFAULT_SIZE 0x300000 > >>> + > >>> struct firmware; > >>> > >>> /* > >>> @@ -192,6 +197,7 @@ struct amdgpu_gmc { > >>> uint32_t srbm_soft_reset; > >>> bool prt_warning; > >>> uint64_t stolen_size; > >>> + bool stolen_temp_reserved; > >>> /* apertures */ > >>> u64 shared_aperture_start; > >>> u64 shared_aperture_end; > >>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c > >>> index 7dc8c068c62a..0c96b67d6ca7 100644 > >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c > >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c > >>> @@ -566,6 +566,11 @@ static int gmc_v10_0_late_init(void *handle) > >>> struct amdgpu_device *adev = (struct amdgpu_device *)handle; > >>> int r; > >>> > >>> + if (adev->gmc.stolen_temp_reserved) { > >>> + amdgpu_bo_late_init(adev); > >>> + adev->gmc.stolen_temp_reserved = false; > >>> + } > >>> + > >>> r = amdgpu_gmc_allocate_vm_inv_eng(adev); > >>> if (r) > >>> return r; > >>> @@ -756,6 +761,22 @@ static int gmc_v10_0_sw_init(void *handle) > >>> return r; > >>> > >>> adev->gmc.stolen_size = gmc_v10_0_get_vbios_fb_size(adev); > >>> + /* > >>> + * In dual GPUs scenario, stolen_size is assigned to zero on the 2nd GPU, > >>> + * then the bottom region of VRAM was allocated as GTT, unfortunately > >>> + * a small region of bottom VRAM was encroached by UMC firmware during > >>> + * GDDR6 BIST training, this cause pagefault. > >>> + * The page fault can be fixed by forcing stolen_size to 3MB, then the bottom > >>> + * region of VRAM was allocated as stolen memory, GTT corruption avoid. > >>> + * The stolen memory of the 2nd GPU will be free in late_init phase, > >>> + * no memory wasted. > >>> + */ > >>> + if (adev->fw_vram_usage.mem_train_support && > >>> + adev->gmc.stolen_size == 0) { > >>> + adev->gmc.stolen_size = AMDGPU_STOLEN_VGA_DEFAULT_SIZE; > >>> + adev->gmc.stolen_temp_reserved = true; > >>> + } else > >>> + adev->gmc.stolen_temp_reserved = false; > >>> > >>> /* Memory manager */ > >>> r = amdgpu_bo_init(adev); > >> _______________________________________________ > >> amd-gfx mailing list > >> amd-gfx@xxxxxxxxxxxxxxxxxxxxx > >> https://nam11.safelinks.protection.outlook.com/?url=""> > > _______________________________________________ > > amd-gfx mailing list > > amd-gfx@xxxxxxxxxxxxxxxxxxxxx > > https://nam11.safelinks.protection.outlook.com/?url=""> > |
_______________________________________________ amd-gfx mailing list amd-gfx@xxxxxxxxxxxxxxxxxxxxx https://lists.freedesktop.org/mailman/listinfo/amd-gfx
