Re: [PATCH 1/2] drm/amdkfd: introduce dummy cache info for property asic

[Felix Kuehling <felix.kuehling@xxxxxxx>]

Am 2022-10-20 um 21:50 schrieb Liang, Prike:
> [Public]
>
> -----Original Message-----
> From: Kuehling, Felix <Felix.Kuehling@xxxxxxx>
> Sent: Friday, October 21, 2022 12:03 AM
> To: Liang, Prike <Prike.Liang@xxxxxxx>; amd-gfx@xxxxxxxxxxxxxxxxxxxxx
> Cc: Deucher, Alexander <Alexander.Deucher@xxxxxxx>; Zhang, Yifan <Yifan1.Zhang@xxxxxxx>; Huang, Ray <Ray.Huang@xxxxxxx>; Liu, Aaron <Aaron.Liu@xxxxxxx>
> Subject: Re: [PATCH 1/2] drm/amdkfd: introduce dummy cache info for property asic
>
>
> Am 2022-10-20 um 05:15 schrieb Prike Liang:
> > This dummy cache info will enable kfd base function support.
> >
> > Signed-off-by: Prike Liang <Prike.Liang@xxxxxxx>
> > ---
> >    drivers/gpu/drm/amd/amdkfd/kfd_crat.c | 55 +++++++++++++++++++++++++--
> >    1 file changed, 52 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_crat.c
> > b/drivers/gpu/drm/amd/amdkfd/kfd_crat.c
> > index cd5f8b219bf9..960046e43b7a 100644
> > --- a/drivers/gpu/drm/amd/amdkfd/kfd_crat.c
> > +++ b/drivers/gpu/drm/amd/amdkfd/kfd_crat.c
> > @@ -795,6 +795,54 @@ static struct kfd_gpu_cache_info yellow_carp_cache_info[] = {
> >        },
> >    };
> >
> > +static struct kfd_gpu_cache_info dummy_cache_info[] = {
> > +     {
> > +             /* TCP L1 Cache per CU */
> > +             .cache_size = 16,
> > +             .cache_level = 1,
> > +             .flags = (CRAT_CACHE_FLAGS_ENABLED |
> > +                             CRAT_CACHE_FLAGS_DATA_CACHE |
> > +                             CRAT_CACHE_FLAGS_SIMD_CACHE),
> > +             .num_cu_shared = 1,
> > +     },
> > +     {
> > +             /* Scalar L1 Instruction Cache per SQC */
> > +             .cache_size = 32,
> > +             .cache_level = 1,
> > +             .flags = (CRAT_CACHE_FLAGS_ENABLED |
> > +                             CRAT_CACHE_FLAGS_INST_CACHE |
> > +                             CRAT_CACHE_FLAGS_SIMD_CACHE),
> > +             .num_cu_shared = 2,
> > +     },
> > +     {
> > +             /* Scalar L1 Data Cache per SQC */
> > +             .cache_size = 16,
> > +             .cache_level = 1,
> > +             .flags = (CRAT_CACHE_FLAGS_ENABLED |
> > +                             CRAT_CACHE_FLAGS_DATA_CACHE |
> > +                             CRAT_CACHE_FLAGS_SIMD_CACHE),
> > +             .num_cu_shared = 2,
> > +     },
> > +     {
> > +             /* GL1 Data Cache per SA */
> > +             .cache_size = 128,
> > +             .cache_level = 1,
> > +             .flags = (CRAT_CACHE_FLAGS_ENABLED |
> > +                             CRAT_CACHE_FLAGS_DATA_CACHE |
> > +                             CRAT_CACHE_FLAGS_SIMD_CACHE),
> > +             .num_cu_shared = 6,
> > +     },
> > +     {
> > +             /* L2 Data Cache per GPU (Total Tex Cache) */
> > +             .cache_size = 2048,
> > +             .cache_level = 2,
> > +             .flags = (CRAT_CACHE_FLAGS_ENABLED |
> > +                             CRAT_CACHE_FLAGS_DATA_CACHE |
> > +                             CRAT_CACHE_FLAGS_SIMD_CACHE),
> > +             .num_cu_shared = 6,
> > +     },
> > +};
> > +
> >    static void kfd_populated_cu_info_cpu(struct kfd_topology_device *dev,
> >                struct crat_subtype_computeunit *cu)
> >    {
> > @@ -1514,8 +1562,6 @@ static int kfd_fill_gpu_cache_info(struct kfd_dev *kdev,
> >                        num_of_cache_types = ARRAY_SIZE(beige_goby_cache_info);
> >                        break;
> >                case IP_VERSION(10, 3, 3):
> > -             case IP_VERSION(10, 3, 6): /* TODO: Double check these on production silicon */
> > -             case IP_VERSION(10, 3, 7): /* TODO: Double check these on production silicon */
> >                        pcache_info = yellow_carp_cache_info;
> >                        num_of_cache_types = ARRAY_SIZE(yellow_carp_cache_info);
> >                        break;
> > @@ -1528,7 +1574,10 @@ static int kfd_fill_gpu_cache_info(struct kfd_dev *kdev,
> >                                kfd_fill_gpu_cache_info_from_gfx_config(kdev, pcache_info);
> >                        break;
> >                default:
> > -                     return -EINVAL;
> > +                     pcache_info = dummy_cache_info;
> > +                     num_of_cache_types = ARRAY_SIZE(dummy_cache_info);
> > +                     pr_warn("dummy cache info is used temporarily and real cache info need update later.\n");
> > +                     break;
> Could we make this return an error if the amdgpu.exp_hw_support module parameter is not set?
>
> Regards,
>     Felix
>
> [Prike] It's fine to protect this dummy info by checking the parameter amdgpu_exp_hw_support, but it may not friendly to end user by adding the parameter and some guys will still report KFD not enabled for this parameter setting problem. The original idea is the end user will not aware the dummy cache info and only alert the warning message to developer.

I thought the intention was to simplify bring-up but make sure that 
valid cache info is available by the time a chip goes into production. 
Therefore, normal end users should never need to set the 
amdgpu_exp_hw_support option. I think you're saying that we would go to 
production with dummy info. That seems like a bad idea to me.

Regards,
  Felix


> >                }
> >        }