[AMD Official Use Only - General] Hi Sanjaya, > -----Original Message----- > From: Bagas Sanjaya <bagasdotme@xxxxxxxxx> > Sent: Friday, October 7, 2022 3:33 AM > To: Moger, Babu <Babu.Moger@xxxxxxx>; corbet@xxxxxxx; > reinette.chatre@xxxxxxxxx; tglx@xxxxxxxxxxxxx; mingo@xxxxxxxxxx; > bp@xxxxxxxxx > Cc: fenghua.yu@xxxxxxxxx; dave.hansen@xxxxxxxxxxxxxxx; x86@xxxxxxxxxx; > hpa@xxxxxxxxx; paulmck@xxxxxxxxxx; akpm@xxxxxxxxxxxxxxxxxxxx; > quic_neeraju@xxxxxxxxxxx; rdunlap@xxxxxxxxxxxxx; > damien.lemoal@xxxxxxxxxxxxxxxxxx; songmuchun@xxxxxxxxxxxxx; > peterz@xxxxxxxxxxxxx; jpoimboe@xxxxxxxxxx; pbonzini@xxxxxxxxxx; > chang.seok.bae@xxxxxxxxx; pawan.kumar.gupta@xxxxxxxxxxxxxxx; > jmattson@xxxxxxxxxx; daniel.sneddon@xxxxxxxxxxxxxxx; Das1, Sandipan > <Sandipan.Das@xxxxxxx>; tony.luck@xxxxxxxxx; james.morse@xxxxxxx; > linux-doc@xxxxxxxxxxxxxxx; linux-kernel@xxxxxxxxxxxxxxx; > eranian@xxxxxxxxxx > Subject: Re: [PATCH v5 00/12] x86/resctrl: Support for AMD QoS new features > > On 9/28/22 03:25, Babu Moger wrote: > > New AMD processors can now support following QoS features. > > > > 1. Slow Memory Bandwidth Allocation (SMBA) > > With this feature, the QOS enforcement policies can be applied > > to the external slow memory connected to the host. QOS enforcement > > is accomplished by assigning a Class Of Service (COS) to a processor > > and specifying allocations or limits for that COS for each resource > > to be allocated. > > > > Currently, CXL.memory is the only supported "slow" memory device. With > > the support of SMBA feature the hardware enables bandwidth allocation > > on the slow memory devices. > > > > 2. Bandwidth Monitoring Event Configuration (BMEC) > > The bandwidth monitoring events mbm_total_event and mbm_local_event > > are set to count all the total and local reads/writes respectively. > > With the introduction of slow memory, the two counters are not enough > > to count all the different types are memory events. With the feature > > BMEC, the users have the option to configure mbm_total_event and > > mbm_local_event to count the specific type of events. > > > > Following are the bitmaps of events supported. > > Bits Description > > 6 Dirty Victims from the QOS domain to all types of memory > > 5 Reads to slow memory in the non-local NUMA domain > > 4 Reads to slow memory in the local NUMA domain > > 3 Non-temporal writes to non-local NUMA domain > > 2 Non-temporal writes to local NUMA domain > > 1 Reads to memory in the non-local NUMA domain > > 0 Reads to memory in the local NUMA domain > > > > This series adds support for these features. > > > > Feature description is available in the specification, "AMD64 > > Technology Platform Quality of Service Extensions, Revision: 1.03 Publication > # 56375 Revision: 1.03 Issue Date: February 2022". > > > > Link: > > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww. > > amd.com%2Fen%2Fsupport%2Ftech-docs%2Famd64-technology-platform- > quality > > -service- > extensions&data=05%7C01%7Cbabu.moger%40amd.com%7Cda5fc806 > > > 9ca2484b2aef08daa83ea08a%7C3dd8961fe4884e608e11a82d994e183d%7C0% > 7C0%7C > > > 638007284215759374%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMD > AiLCJQIjo > > > iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdat > a=qAZS > > ze2Mbg24Z0%2BX0GN4yrVO2ooQqQyum7NUjwIGg5o%3D&reserved=0 > > Link: > > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugz > > > illa.kernel.org%2Fshow_bug.cgi%3Fid%3D206537&data=05%7C01%7Cbab > u.m > > > oger%40amd.com%7Cda5fc8069ca2484b2aef08daa83ea08a%7C3dd8961fe488 > 4e608e > > > 11a82d994e183d%7C0%7C0%7C638007284215759374%7CUnknown%7CTWFpb > GZsb3d8ey > > > JWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7 > C300 > > > 0%7C%7C%7C&sdata=8%2BFbGTVfkp%2BCbyyk%2BYa9u0JiHi2YZEVaHiUs > CBw335g > > %3D&reserved=0 > > > > --- > > v5: > > Summary of changes. > > 1. Split the series into two. The first two patches are bug fixes. So, sent them > separate. > > 2. The config files mbm_total_config and mbm_local_config are now under > > /sys/fs/resctrl/info/L3_MON/. Removed these config files from mon > groups. > > 3. Ran "checkpatch --strict --codespell" on all the patches. Looks good with > few known exceptions. > > 4. Few minor text changes in resctrl.rst file. > > > > v4: > > > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kern > el.org%2Flkml%2F166257348081.1043018.11227924488792315932.stgit%40bm > oger- > ubuntu%2F&data=05%7C01%7Cbabu.moger%40amd.com%7Cda5fc8069ca > 2484b2aef08daa83ea08a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0 > %7C638007284215759374%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw > MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7 > C&sdata=qamR1M2sVSo4vE9NVZzFmvIop7YMKNIYHT74NJrbVVk%3D&am > p;reserved=0 > > Got numerios of comments from Reinette Chatre. Addressed most of them. > > Summary of changes. > > 1. Removed mon_configurable under /sys/fs/resctrl/info/L3_MON/. > > 2. Updated mon_features texts if the BMEC is supported. > > 3. Added more explanation about the slow memory support. > > 4. Replaced smp_call_function_many with on_each_cpu_mask call. > > 5. Removed arch_has_empty_bitmaps > > 6. Few other text changes. > > 7. Removed Reviewed-by if the patch is modified. > > 8. Rebased the patches to latest tip. > > > > v3: > > > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kern > el.org%2Flkml%2F166117559756.6695.16047463526634290701.stgit%40bmoge > r- > ubuntu%2F&data=05%7C01%7Cbabu.moger%40amd.com%7Cda5fc8069ca > 2484b2aef08daa83ea08a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0 > %7C638007284215915604%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw > MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7 > C&sdata=ogINZslT9yExkkFww4X14XQEg8W8heYBrJB59C50Hqk%3D& > reserved=0 > > a. Rebased the patches to latest tip. Resolved some conflicts. > > > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kerne > l.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftip%2Ftip.git&data=05% > 7C01%7Cbabu.moger%40amd.com%7Cda5fc8069ca2484b2aef08daa83ea08a% > 7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638007284215915604 > %7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJ > BTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ZOK3iwsaq > 3%2BGAUIXJwn9Thg3cJBxWDMAfu4UqHlo2J4%3D&reserved=0 > > b. Taken care of feedback from Bagas Sanjaya. > > c. Added Reviewed by from Mingo. > > Note: I am still looking for comments from Reinette or Fenghua. > > > > v2: > > > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kern > el.org%2Flkml%2F165938717220.724959.10931629283087443782.stgit%40bmo > ger- > ubuntu%2F&data=05%7C01%7Cbabu.moger%40amd.com%7Cda5fc8069ca > 2484b2aef08daa83ea08a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0 > %7C638007284215915604%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw > MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7 > C&sdata=14viyG9elsn6BYGpDOwrqYQFNOlpyC6oqoJwJm49oV0%3D& > reserved=0 > > a. Rebased the patches to latest stable tree (v5.18.15). Resolved some > conflicts. > > b. Added the patch to fix CBM issue on AMD. This was originally discussed > > > > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore > > .kernel.org%2Flkml%2F20220517001234.3137157-1- > eranian%40google.com%2F& > > > amp;data=05%7C01%7Cbabu.moger%40amd.com%7Cda5fc8069ca2484b2aef0 > 8daa83e > > > a08a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C6380072842159 > 15604%7 > > > CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI > 6Ik1 > > > haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2BHEvN2PNNYyH > ohvLg2sbth > > BQo4cgDj5Vsw9AqGb1Pr8%3D&reserved=0 > > > > v1: > > > > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore > > > .kernel.org%2Flkml%2F165757543252.416408.13547339307237713464.stgit%4 > 0 > > bmoger- > ubuntu%2F&data=05%7C01%7Cbabu.moger%40amd.com%7Cda5fc8069ca > > > 2484b2aef08daa83ea08a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0 > %7C638 > > > 007284215915604%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLC > JQIjoiV2 > > > luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=J > R03cAA > > 9pzmq0SgNsgAsajOH6LX%2F4s3px3b%2FJ8409Ys%3D&reserved=0 > > > > Babu Moger (12): > > x86/cpufeatures: Add Slow Memory Bandwidth Allocation feature flag > > x86/resctrl: Add a new resource type RDT_RESOURCE_SMBA > > x86/cpufeatures: Add Bandwidth Monitoring Event Configuration feature > flag > > x86/resctrl: Include new features in command line options > > x86/resctrl: Detect and configure Slow Memory Bandwidth allocation > > x86/resctrl: Introduce data structure to support monitor configuration > > x86/resctrl: Add sysfs interface to read mbm_total_bytes event > configuration > > x86/resctrl: Add sysfs interface to read mbm_local_bytes event > configuration > > x86/resctrl: Add sysfs interface to write mbm_total_bytes event > configuration > > x86/resctrl: Add sysfs interface to write mbm_local_bytes event > configuration > > x86/resctrl: Replace smp_call_function_many() with on_each_cpu_mask() > > Documentation/x86: Update resctrl_ui.rst for new features > > > > > > .../admin-guide/kernel-parameters.txt | 2 +- > > Documentation/x86/resctrl.rst | 130 +++++++- > > arch/x86/include/asm/cpufeatures.h | 2 + > > arch/x86/kernel/cpu/cpuid-deps.c | 1 + > > arch/x86/kernel/cpu/resctrl/core.c | 51 ++- > > arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 2 +- > > arch/x86/kernel/cpu/resctrl/internal.h | 33 +- > > arch/x86/kernel/cpu/resctrl/monitor.c | 9 +- > > arch/x86/kernel/cpu/resctrl/rdtgroup.c | 298 ++++++++++++++++-- > > arch/x86/kernel/cpu/scattered.c | 2 + > > 10 files changed, 496 insertions(+), 34 deletions(-) > > > > Hi Babu, sorry for having to do public reply to this v5 cover letter due to > accidentally delete the preview documentation patch for your upcoming v6. > > Thanks for privately sending me the preview patch. Seeing it at a glance, > LGTM. Please send the full v6 series for us to review. Sure. Will send the whole series. Thanks Babu
