This series adds the support for Assignable Bandwidth Monitoring Counters (ABMC). It is also called QoS RMID Pinning feature Series is written such that it is easier to support other assignable features supported from different vendors. The feature details are documented in the APM listed below [1]. [1] AMD64 Architecture Programmer's Manual Volume 2: System Programming Publication # 24593 Revision 3.41 section 19.3.3.3 Assignable Bandwidth Monitoring (ABMC). The documentation is available at Link: https://bugzilla.kernel.org/show_bug.cgi?id=206537 The patches are based on top of commit d803bbbb55f4f (tip/master) Merge branch into tip/master: 'x86/tdx' # Introduction Users can create as many monitor groups as RMIDs supported by the hardware. However, bandwidth monitoring feature on AMD system only guarantees that RMIDs currently assigned to a processor will be tracked by hardware. The counters of any other RMIDs which are no longer being tracked will be reset to zero. The MBM event counters return "Unavailable" for the RMIDs that are not tracked by hardware. So, there can be only limited number of groups that can give guaranteed monitoring numbers. With ever changing configurations there is no way to definitely know which of these groups are being tracked for certain point of time. Users do not have the option to monitor a group or set of groups for certain period of time without worrying about counter being reset in between. The ABMC feature provides an option to the user to assign a hardware counter to an RMID, event pair and monitor the bandwidth as long as it is assigned. The assigned RMID will be tracked by the hardware until the user unassigns it manually. There is no need to worry about counters being reset during this period. Additionally, the user can specify a bitmask identifying the specific bandwidth types from the given source to track with the counter. Without ABMC enabled, monitoring will work in current 'default' mode without assignment option. # Linux Implementation Create a generic interface aimed to support user space assignment of scarce counters used for monitoring. First usage of interface is by ABMC with option to expand usage to "soft-ABMC" and MPAM counters in future. Feature adds following interface files: /sys/fs/resctrl/info/L3_MON/mbm_assign_mode: Reports the list of assignable monitoring features supported. The enclosed brackets indicate which feature is enabled. /sys/fs/resctrl/info/L3_MON/num_mbm_cntrs: Reports the number of monitoring counters available for assignment. /sys/fs/resctrl/info/L3_MON/available_mbm_cntrs: Reports the number of monitoring counters free in each domain. /sys/fs/resctrl/info/L3_MON/mbm_assign_control: Reports the resctrl group and monitor status of each group. Assignment state can be updated by writing to the interface. # Examples a. Check if ABMC support is available #mount -t resctrl resctrl /sys/fs/resctrl/ # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode [mbm_cntr_assign] default ABMC feature is detected and it is enabled. b. Check how many ABMC counters are available. # cat /sys/fs/resctrl/info/L3_MON/num_mbm_cntrs 32 c. Check how many ABMC counters are available in each domain. # cat /sys/fs/resctrl/info/L3_MON/available_mbm_cntrs 0=30;1=30; d. Create few resctrl groups. # mkdir /sys/fs/resctrl/mon_groups/child_default_mon_grp # mkdir /sys/fs/resctrl/non_default_ctrl_mon_grp # mkdir /sys/fs/resctrl/non_default_ctrl_mon_grp/mon_groups/child_non_default_mon_grp e. This series adds a new interface file /sys/fs/resctrl/info/L3_MON/mbm_assign_control to list and modify any group's monitoring states. File provides single place to list monitoring states of all the resctrl groups. It makes it easier for user space to learn about the used counters without needing to traverse all the groups thus reducing the number of file system calls. The list follows the following format: "<CTRL_MON group>/<MON group>/<domain_id>=<flags>" Format for specific type of groups: * Default CTRL_MON group: "//<domain_id>=<flags>" * Non-default CTRL_MON group: "<CTRL_MON group>//<domain_id>=<flags>" * Child MON group of default CTRL_MON group: "/<MON group>/<domain_id>=<flags>" * Child MON group of non-default CTRL_MON group: "<CTRL_MON group>/<MON group>/<domain_id>=<flags>" Flags can be one of the following: t MBM total event is enabled. l MBM local event is enabled. tl Both total and local MBM events are enabled. _ None of the MBM events are enabled Examples: # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_control non_default_ctrl_mon_grp//0=tl;1=tl; non_default_ctrl_mon_grp/child_non_default_mon_grp/0=tl;1=tl; //0=tl;1=tl; /child_default_mon_grp/0=tl;1=tl; There are four groups and all the groups have local and total event enabled on domain 0 and 1. f. Update the group assignment states using the interface file /sys/fs/resctrl/info/L3_MON/mbm_assign_control. The write format is similar to the above list format with addition of opcode for the assignment operation. “<CTRL_MON group>/<MON group>/<domain_id><opcode><flags>” * Default CTRL_MON group: "//<domain_id><opcode><flags>" * Non-default CTRL_MON group: "<CTRL_MON group>//<domain_id><opcode><flags>" * Child MON group of default CTRL_MON group: "/<MON group>/<domain_id><opcode><flags>" * Child MON group of non-default CTRL_MON group: "<CTRL_MON group>/<MON group>/<domain_id><opcode><flags>" Opcode can be one of the following: = Update the assignment to match the flags. + Assign a new MBM event without impacting existing assignments. - Unassign a MBM event from currently assigned events. Flags can be one of the following: t MBM total event. l MBM local event. tl Both total and local MBM events. _ None of the MBM events. Only works with '=' opcode. This flag cannot be combined with other flags. Initial group status: # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_control non_default_ctrl_mon_grp//0=tl;1=tl; non_default_ctrl_mon_grp/child_non_default_mon_grp/0=tl;1=tl; //0=tl;1=tl; /child_default_mon_grp/0=tl;1=tl; To update the default group to enable only total event on domain 0: # echo "//0=t" > /sys/fs/resctrl/info/L3_MON/mbm_assign_control Assignment status after the update: # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_control non_default_ctrl_mon_grp//0=tl;1=tl; non_default_ctrl_mon_grp/child_non_default_mon_grp/0=tl;1=tl; //0=t;1=tl; /child_default_mon_grp/0=tl;1=tl; To update the MON group child_default_mon_grp to remove total event on domain 1: # echo "/child_default_mon_grp/1-t" > /sys/fs/resctrl/info/L3_MON/mbm_assign_control Assignment status after the update: $ cat /sys/fs/resctrl/info/L3_MON/mbm_assign_control non_default_ctrl_mon_grp//0=tl;1=tl; non_default_ctrl_mon_grp/child_non_default_mon_grp/0=tl;1=tl; //0=t;1=tl; /child_default_mon_grp/0=tl;1=l; To update the MON group non_default_ctrl_mon_grp/child_non_default_mon_grp to remove both local and total events on domain 1: # echo "non_default_ctrl_mon_grp/child_non_default_mon_grp/1=_" > /sys/fs/resctrl/info/L3_MON/mbm_assign_control Assignment status after the update: $ cat /sys/fs/resctrl/info/L3_MON/mbm_assign_control non_default_ctrl_mon_grp//0=tl;1=tl; non_default_ctrl_mon_grp/child_non_default_mon_grp/0=tl;1=_; //0=t;1=tl; /child_default_mon_grp/0=tl;1=l; To update the default group to add a local event domain 0. # echo "//0+l" > /sys/fs/resctrl/info/L3_MON/mbm_assign_control Assignment status after the update: # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_control non_default_ctrl_mon_grp//0=tl;1=tl; non_default_ctrl_mon_grp/child_non_default_mon_grp/0=tl;1=_; //0=tl;1=tl; /child_default_mon_grp/0=tl;1=l; To update the non default CTRL_MON group non_default_ctrl_mon_grp to unassign all the MBM events on all the domains. # echo "non_default_ctrl_mon_grp//*=_" > /sys/fs/resctrl/info/L3_MON/mbm_assign_control Assignment status after the update: # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_control non_default_ctrl_mon_grp//0=_;1=_; non_default_ctrl_mon_grp/child_non_default_mon_grp/0=tl;1=_; //0=tl;1=tl; /child_default_mon_grp/0=tl;1=l; g. Read the event mbm_total_bytes and mbm_local_bytes of the default group. There is no change in reading the events with ABMC. If the event is unassigned when reading, then the read will come back as "Unassigned". # cat /sys/fs/resctrl/mon_data/mon_L3_00/mbm_total_bytes 779247936 # cat /sys/fs/resctrl/mon_data/mon_L3_00/mbm_local_bytes 765207488 h. Check the bandwidth configuration for the group. Note that bandwidth configuration has a domain scope. Total event defaults to 0x7F (to count all the events) and local event defaults to 0x15 (to count all the local numa events). The event bitmap decoding is available at https://www.kernel.org/doc/Documentation/x86/resctrl.rst in section "mbm_total_bytes_config", "mbm_local_bytes_config": #cat /sys/fs/resctrl/info/L3_MON/mbm_total_bytes_config 0=0x7f;1=0x7f #cat /sys/fs/resctrl/info/L3_MON/mbm_local_bytes_config 0=0x15;1=0x15 i. Change the bandwidth source for domain 0 for the total event to count only reads. Note that this change effects total events on the domain 0. #echo 0=0x33 > /sys/fs/resctrl/info/L3_MON/mbm_total_bytes_config #cat /sys/fs/resctrl/info/L3_MON/mbm_total_bytes_config 0=0x33;1=0x7F j. Now read the total event again. The first read will come back with "Unavailable" status. The subsequent read of mbm_total_bytes will display only the read events. #cat /sys/fs/resctrl/mon_data/mon_L3_00/mbm_total_bytes Unavailable #cat /sys/fs/resctrl/mon_data/mon_L3_00/mbm_total_bytes 314101 k. Users will have the option to go back to 'default' mbm_assign_mode if required. This can be done using the following command. Note that switching the mbm_assign_mode will reset all the MBM counters of all resctrl groups. # echo "default" > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode mbm_cntr_assign [default] l. Unmount the resctrl #umount /sys/fs/resctrl/ --- v10: Major change is related to domain specific assignment. Added struct mbm_cntr_cfg inside mon domains. This will handle the domain specific assignments as discussed in below. https://lore.kernel.org/lkml/CALPaoCj+zWq1vkHVbXYP0znJbe6Ke3PXPWjtri5AFgD9cQDCUg@xxxxxxxxxxxxxx/ I did not see the need to add cntr_id in mbm_state structure. Not used in the code. Following patches take care of these changes. Patch 12, 13, 15, 16, 17, 18. Added __init attribute to cache_alloc_hsw_probe(). Followed function prototype rules (preferred order is storage class before return type). Moved the mon_config_info structure definition to resctrl.h Added call resctrl_arch_reset_rmid() to reset the RMID in the domain inside IPI call resctrl_abmc_config_one_amd. SMP and non-SMP call support is not required in resctrl_arch_config_cntr with new domain specific assign approach/data structure. Assigned the counter before exposing the event files. Moved the call rdtgroup_assign_cntrs() inside mkdir_rdt_prepare_rmid_alloc(). This is called both CNTR_MON and MON group creation. Call mbm_cntr_reset() when unmounted to clear all the assignments. Fixed the issue with finding the domain in multiple iterations in rdtgroup_process_flags(). Printed full error message with domain information when assign fails. Taken care of other text comments in all the patches. Patch specific changes are in each patch. If I missed something please point me and it is not intentional. v9: Patch 14 is a new addition. Major change in patch 24. Moved the fix patch to address __init attribute to begining of the series. Fixed all the call sequences. Added additional Fixed tags. Added Reviewed-by where applicable. Took care of couple of minor merge conflicts with latest code. Re-ordered the MSR in couple of instances. Added available_mbm_cntrs (patch 14) to print the number of counter in a domain. Used MBM_EVENT_ARRAY_INDEX macro to get the event index. Introduced rdtgroup_cntr_id_init() to initialize the cntr_id Introduced new function resctrl_config_cntr to assign the counter, update the bitmap and reset the architectural state. Taken care of error handling(freeing the counter) when assignment fails. Changed rdtgroup_assign_cntrs() and rdtgroup_unassign_cntrs() to return void. Updated couple of rdtgroup_unassign_cntrs() calls properly. Fixed problem changing the mode to mbm_cntr_assign mode when it is not supported. Added extra checks to detect if systems supports it. https://lore.kernel.org/lkml/03b278b5-6c15-4d09-9ab7-3317e84a409e@xxxxxxxxx/ As discussed in the above comment, introduced resctrl_mon_event_config_set to handle IPI. But sending another IPI inside IPI causes problem. Kernel reports SMP warning. So, introduced resctrl_arch_update_cntr() to send the command directly. Fixed handling special case '//0=' and '//". Removed extra strstr() call in rdtgroup_mbm_assign_control_write(). Added generic failure text when assignment operation fails. Corrected user documentation format texts. v8: Patches are getting into final stages. Couple of changes Patch 8, Patch 19 and Patch 23. Most of the other changes are related to rename and text message updates. Details are in each patch. Here is the summary. Added __init attribute to dom_data_init() in patch 8/25. Moved the mbm_cntrs_init() and mbm_cntrs_exit() functionality inside dom_data_init() and dom_data_exit() respectively. Renamed resctrl_mbm_evt_config_init() to arch_mbm_evt_config_init() Renamed resctrl_arch_event_config_get() to resctrl_arch_mon_event_config_get(). resctrl_arch_event_config_set() to resctrl_arch_mon_event_config_set(). Rename resctrl_arch_assign_cntr to resctrl_arch_config_cntr. Renamed rdtgroup_assign_cntr() to rdtgroup_assign_cntr_event(). Added the code to return the error if rdtgroup_assign_cntr_event fails. Moved definition of MBM_EVENT_ARRAY_INDEX to resctrl/internal.h. Renamed rdtgroup_mbm_cntr_is_assigned to mbm_cntr_assigned_to_domain Added return error handling in resctrl_arch_config_cntr(). Renamed rdtgroup_assign_grp to rdtgroup_assign_cntrs. Renamed rdtgroup_unassign_grp to rdtgroup_unassign_cntrs. Fixed the problem with unassigning the child MON groups of CTRL_MON group. Reset the internal counters after mbm_cntr_assign mode is changed. Renamed rdtgroup_mbm_cntr_reset() to mbm_cntr_reset() Renamed resctrl_arch_mbm_cntr_assign_configure to resctrl_arch_mbm_cntr_assign_set_one. Used the same IPI as event update to modify the assignment. Could not do the way we discussed in the thread. https://lore.kernel.org/lkml/f77737ac-d3f6-3e4b-3565-564f79c86ca8@xxxxxxx/ Needed to figure out event type to update the configuration. Moved unassign first and assign during the assign modification. Assign none "_" takes priority. Cannot be mixed with other flags. Updated the documentation and .rst file format. htmldoc looks ok. v7: Major changes are related to FS and arch codes separation. Changed few interface names based on feedback. Here are the summary and each patch contains changes specific the patch. Removed WARN_ON for num_mbm_cntrs. Decided to dynamically allocate the bitmap. WARN_ON is not required anymore. Renamed the function resctrl_arch_get_abmc_enabled() to resctrl_arch_mbm_cntr_assign_enabled(). Merged resctrl_arch_mbm_cntr_assign_disable, resctrl_arch_mbm_cntr_assign_disable and renamed to resctrl_arch_mbm_cntr_assign_set(). Passed the struct rdt_resource to these functions. Removed resctrl_arch_reset_rmid_all() from arch code. This will be done from FS the caller. Updated the descriptions/commit log in resctrl.rst to generic text. Removed ABMC references. Renamed mbm_mode to mbm_assign_mode. Renamed mbm_control to mbm_assign_control. Introduced mutex lock in rdtgroup_mbm_mode_show(). The 'legacy' mode is called 'default' mode. Removed the static allocation and now allocating bitmap mbm_cntr_free_map dynamically. Merged rdtgroup_assign_cntr(), rdtgroup_alloc_cntr() into one. Merged rdtgroup_unassign_cntr(), rdtgroup_free_cntr() into one. Added struct rdt_resource to the interface functions resctrl_arch_assign_cntr () and resctrl_arch_unassign_cntr(). Rename rdtgroup_abmc_cfg() to resctrl_abmc_config_one_amd(). Added a new patch to fix counter assignment on event config changes. Removed the references of ABMC from user interfaces. Simplified the parsing (strsep(&token, "//") in rdtgroup_mbm_assign_control_write(). Added mutex lock in rdtgroup_mbm_assign_control_write() while processing. Thomas Gleixner asked us to update https://gitlab.com/x86-cpuid.org/x86-cpuid-db. It needs internal approval. We are working on it. v6: We still need to finalize few interface details on mbm_assign_mode and mbm_assign_control in case of ABMC and Soft-ABMC. We can continue the discussion with this series. Added support for domain-id '*' to update all the domains at once. Fixed assign interface to allocate the counter if counter is not assigned. Fixed unassign interface to free the counter if the counter is not assigned in any of the domains. Renamed abmc_capable to mbm_cntr_assignable. Renamed abmc_enabled to mbm_cntr_assign_enabled. Used msr_set_bit and msr_clear_bit for msr updates. Renamed resctrl_arch_abmc_enable() to resctrl_arch_mbm_cntr_assign_enable(). Renamed resctrl_arch_abmc_disable() to resctrl_arch_mbm_cntr_assign_disable(). Changed the display name from num_cntrs to num_mbm_cntrs. Removed the variable mbm_cntrs_free_map_len. This is not required. Removed the call mbm_cntrs_init() in arch code. This needs to be done at higher level. Used DECLARE_BITMAP to initialize mbm_cntrs_free_map. Removed unused config value definitions. Introduced mbm_cntr_map to track counters at domain level. With this we dont need to send MSR read to read the counter configuration. Separated all the counter id management to upper level in FS code. Added checks to detect "Unassigned" before reading the RMID. More details in each patch. v5: Rebase changes (because of SNC support) Interface changes. /sys/fs/resctrl/mbm_assign to /sys/fs/resctrl/mbm_assign_mode. /sys/fs/resctrl/mbm_assign_control to /sys/fs/resctrl/mbm_assign_control. Added few arch specific routines. resctrl_arch_get_abmc_enabled. resctrl_arch_abmc_enable. resctrl_arch_abmc_disable. Few renames num_cntrs_free_map -> mbm_cntrs_free_map num_cntrs_init -> mbm_cntrs_init arch_domain_mbm_evt_config -> resctrl_arch_mbm_evt_config Introduced resctrl_arch_event_config_get and resctrl_arch_event_config_set() to update event configuration. Removed mon_state field mongroup. Added MON_CNTR_UNSET to initialize counters. Renamed ctr_id to cntr_id for the hardware counter. Report "Unassigned" in case the user attempts to read the events without assigning the counter. ABMC is enabled during the boot up. Can be enabled or disabled later. Fixed opcode and flags combination. '=_" is valid. "-_" amd "+_" is not valid. Added all the comments as far as I know. If I missed something, it is not intentional. v4: Main change is domain specific event assignment. Kept the ABMC feature as a default. Dynamcic switching between ABMC and mbm_legacy is still allowed. We are still not clear about mount option. Moved the monitoring related data in resctrl_mon structure from rdt_resource. Fixed the display of legacy and ABMC mode. Used bimap APIs when possible. Removed event configuration read from MSRs. We can use the internal saved data.(patch 12) Added more comments about L3_QOS_ABMC_CFG MSR. Added IPIs to read the assignment status for each domain (patch 18 and 19) More details in each patch. v3: This series adds the support for global assignment mode discussed in the thread. https://lore.kernel.org/lkml/20231201005720.235639-1-babu.moger@xxxxxxx/ Removed the individual assignment mode and included the global assignment interface. Added following interface files. a. /sys/fs/resctrl/info/L3_MON/mbm_assign Used for displaying the current assignment mode and switch between ABMC and legacy mode. b. /sys/fs/resctrl/info/L3_MON/mbm_assign_control Used for lising the groups assignment mode and modify the assignment states. c. Most of the changes are related to the new interface. d. Addressed the comments from Reinette, James and Peter. e. Hope I have addressed most of the major feedbacks discussed. If I missed something then it is not intentional. Please feel free to comment. f. Sending this as an RFC as per Reinette's comment. So, this is still open for discussion. v2: a. Major change is the way ABMC is enabled. Earlier, user needed to remount with -o abmc to enable ABMC feature. Removed that option now. Now users can enable ABMC by "$echo 1 to /sys/fs/resctrl/info/L3_MON/mbm_assign_enable". b. Added new word 21 to x86/cpufeatures.h. c. Display unsupported if user attempts to read the events when ABMC is enabled and event is not assigned. d. Display monitor_state as "Unsupported" when ABMC is disabled. e. Text updates and rebase to latest tip tree (as of Jan 18). f. This series is still work in progress. I am yet to hear from ARM developers. v9: https://lore.kernel.org/lkml/cover.1730244116.git.babu.moger@xxxxxxx/ v8: https://lore.kernel.org/lkml/cover.1728495588.git.babu.moger@xxxxxxx/ v7: https://lore.kernel.org/lkml/cover.1725488488.git.babu.moger@xxxxxxx/ v6: https://lore.kernel.org/lkml/cover.1722981659.git.babu.moger@xxxxxxx/ v5: https://lore.kernel.org/lkml/cover.1720043311.git.babu.moger@xxxxxxx/ v4: https://lore.kernel.org/lkml/cover.1716552602.git.babu.moger@xxxxxxx/ v3: https://lore.kernel.org/lkml/cover.1711674410.git.babu.moger@xxxxxxx/ v2: https://lore.kernel.org/lkml/20231201005720.235639-1-babu.moger@xxxxxxx/ v1: https://lore.kernel.org/lkml/20231201005720.235639-1-babu.moger@xxxxxxx/ Babu Moger (24): x86/resctrl: Add __init attribute to functions called from resctrl_late_init() x86/cpufeatures: Add support for Assignable Bandwidth Monitoring Counters (ABMC) x86/resctrl: Add ABMC feature in the command line options x86/resctrl: Consolidate monitoring related data from rdt_resource x86/resctrl: Detect Assignable Bandwidth Monitoring feature details x86/resctrl: Introduce resctrl_file_fflags_init() to initialize fflags x86/resctrl: Add support to enable/disable AMD ABMC feature x86/resctrl: Introduce the interface to display monitor mode x86/resctrl: Introduce interface to display number of monitoring counters x86/resctrl: Introduce mbm_total_cfg and mbm_local_cfg in struct rdt_hw_mon_domain x86/resctrl: Remove MSR reading of event configuration value x86/resctrl: Introduce cntr_cfg to track assignable counters at domain x86/resctrl: Introduce interface to display number of free counters x86/resctrl: Add data structures and definitions for ABMC assignment x86/resctrl: Implement resctrl_arch_config_cntr() to assign a counter with ABMC x86/resctrl: Add interface to the assign counter x86/resctrl: Add the interface to unassign a counter x86/resctrl: Auto assign/unassign counters when mbm_cntr_assign is enabled x86/resctrl: Report "Unassigned" for MBM events in mbm_cntr_assign mode x86/resctrl: Introduce the interface to switch between monitor modes x86/resctrl: Configure mbm_cntr_assign mode if supported x86/resctrl: Update assignments on event configuration changes x86/resctrl: Introduce interface to list assignment states of all the groups x86/resctrl: Introduce interface to modify assignment states of the groups .../admin-guide/kernel-parameters.txt | 2 +- Documentation/arch/x86/resctrl.rst | 233 +++++ arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/msr-index.h | 2 + arch/x86/kernel/cpu/cpuid-deps.c | 3 + arch/x86/kernel/cpu/resctrl/core.c | 27 +- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 8 + arch/x86/kernel/cpu/resctrl/internal.h | 78 +- arch/x86/kernel/cpu/resctrl/monitor.c | 68 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 971 ++++++++++++++++-- arch/x86/kernel/cpu/scattered.c | 1 + include/linux/resctrl.h | 53 +- 12 files changed, 1355 insertions(+), 92 deletions(-) -- 2.34.1
