Re: [PATCH v2 RESEND 00/17] Introduce RDT memory bandwidth allocation support

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




On 07/29/2018 11:12 PM, bing.niu@xxxxxxxxx wrote:
> From: Bing Niu <bing.niu@xxxxxxxxx>
> 
> This series is to introduce RDT memory bandwidth allocation support by extending
> current virresctrl implementation.
> 
> The Memory Bandwidth Allocation (MBA) feature provides indirect and approximate
> control over memory bandwidth available per-core. This feature provides a method to
> control applications which may be over-utilizing bandwidth relative to their priority
> in environments such as the data-center. The details can be found in Intel's SDM 17.19.7.
> Kernel supports MBA through resctrl file system same as CAT. Each resctrl group have a
> MB parameter to control how much memory bandwidth it can utilize in unit of percentage.
> 
> In this series, MBA is enabled by enhancing existing virresctrl implementation. The
> policy employed for MBA is similar with CAT: The sum of each MBA group's bandwidth
> dose not exceed 100%.
> 
> The enhancement of virresctrl include two main parts:
> 
> Part 1:  Add two new structures virResctrlInfoMemMB and virResctrlAllocMemBW for collecting
>          host system MBA capability and domain memory bandwidth allocation. Those two
>          structures are the extension of existing virResctrlInfo and virResctrlAlloc. With
>          them, virresctrl framework can support MBA and CAT concurrently. Each virResctrlAlloc
>          represent a resource allocation including CAT, or MBA, or CAT&MBA. The policy of MBA is
>          that: total memory bandwidth of each resctrl group, created by virresctrl, does not
>          exceed to 100%.
> 
> Part 2:  On XML part, add new elements to host capabilities query and domain allocation to support
>          memory bandwidth allocation.
>      ---------------------------------------------------------------------------------------------
>          For host capabilities XML, new XML format like below example,
>            <host>
>             .....
>              <memory_bandwidth>
>                <node id='0' cpus='0-19'>
>                  <control granularity='10' min ='10' maxAllocs='8'/>
>                </node>
>              </memory_bandwidth>
>            </host>
> 
>            granularity --- memory bandwidth granularity
>            min         --- minimum memory bandwidth allowed
>            maxAllocs   --- maximum concurrent memory bandwidth allocation allowed.
> 
>      ---------------------------------------------------------------------------------------------
>          For domain XML, new format as below example
>            <domain type='kvm' id='2'>
>              ......
>              <cputune>
>                ......
>                <shares>1024</shares>
>                <memorytune vcpus='0-1'>
>                  <node id='0' bandwidth='20'/>
>                </memorytune>
>              </cputune>
>              ......
>            </domain>
> 
>           id         --- node where memory bandwidth allocation will happen
>           bandwidth  --- bandwidth allocated in percentage
> 
>      ----------------------------------------------------------------------------------------------
> 
> With this extension of the virresctrl, the overall working follow of CAT and MBA is described by below
> picture. XML parser will aggregate MBA and CAT configuration and represents it in one virresctrl object.
> The methods of virresctrl class will manipulate resctrl interface to allocate corresponding resources.
> 
> 
>      <memorytune cpus='0-3'>
>                     +---------+
>                               |    <cachetune vcpus='0-3'>
>         XML                   |           +
>        parser                 +-----------+
>                               |
>                               |
>                  +------------------------------+
>                               |
>                               |
> internal object        +------v--------------+  
> virResctrlAlloc        |   backing object    |
>                        +------+--------------+
>                               |
>                               |
>                  +------------------------------+
>                               |
>                            +--v-------+
>                            |          |
>                            | schemata |
>  /sys/fs/resctrl           | tasks    |
>                            |   .      |
>                            |   .      |
>                            |          |
>                            +----------+
> ---------------------------------------------------------------------
> 
> previous versions and discussion can be found at
>     v1: https://www.redhat.com/archives/libvir-list/2018-July/msg01144.html
> RFC v2: https://www.redhat.com/archives/libvir-list/2018-June/msg01268.html
> RFC v1: https://www.redhat.com/archives/libvir-list/2018-May/msg02101.html
> 
> Changelog:
>        v1 -> this: John's comment: 1. Split calculation of number of memory bandwidth control
>                                       to one patch.
>                                    2. Split virResctrlAllocMemBW relating methods to 5 patch, each
>                                       provides one kind of function, eg: schemata processing, memory
>                                       bandwidth calculation.....
>                                    3. Use resctrl to replace cachetune in domain conf.
>                                    4. Split refactor virDomainCachetuneDefParse into 3 patches. And
>                                       adjust some logic, eg: use %s format error log, renaming
>                                       functions.....
>                                    5. Complete doc description. eg: update cachetune part about vcpus
>                                       overlapping with memorytune, update libvirt version info for memory
>                                       bandwidth control availability.
>                                    6. Some coding style fix.
> 
>        RFC_v2->v1: John's comment: 1. use name MemBW to replace MB for a more clear description.
>                                    2. split rename patch and put refactor function part separately.
>                                    3. split virResctrlInfoMemMB and virResctrlAllocMemBW to different
>                                       patches.
>                                    4. add docs/schemas/*.rng for XML related patches.
>                                    5. some cleanup for coding conventions.
>        RFC_ v1->RFC_v2:
>             Jano's comment: 1. put renaming parts into separated patches.
>                             2. set the initial return value as -1.
>                             3. using full name in structure definition.
>                             4. do not use VIR_CACHE_TYPE_LAST for memory bandwidth allocation formatting.
> 
>             Pavel's comment: 1. add host capabilities XML for memory bandwidth allocation.
>                              2. do not mix use cachetune section in XML for memory bandwidth allocation in
>                                 domain XML. define a dedicated one for memory bandwidth allocation.
> 
> Bing Niu (17):
>   util: Rename some functions of virresctrl
>   util: Refactor virResctrlGetInfo in virresctrl
>   util: Refactor virResctrlAllocFormat of virresctrl
>   util: Add MBA capability information query to resctrl
>   util: Add MBA check to virResctrlInfoGetCache
>   util: Add MBA allocation to virresctrl
>   util: Add MBA schemata parse and format methods
>   util: Add support to calculate MBA utilization
>   util: Introduce virResctrlAllocForeachMemory
>   util: Introduce virResctrlAllocSetMemoryBandwidth
>   conf: Rename cachetune to resctrl
>   conf: Factor out vcpus parsing part from virDomainCachetuneDefParse
>   conf: Factor out vcpus overlapping from virDomainCachetuneDefParse
>   conf: Factor out virDomainResctrlDef update from
>     virDomainCachetuneDefParse
>   conf: Add support for memorytune XML processing for resctrl MBA
>   conf: Add return value check to virResctrlAllocForeachCache
>   conf: Add memory bandwidth allocation capability of host
> 
>  docs/formatdomain.html.in                          |  39 +-
>  docs/schemas/capability.rng                        |  33 ++
>  docs/schemas/domaincommon.rng                      |  17 +
>  src/conf/capabilities.c                            | 107 ++++
>  src/conf/capabilities.h                            |  11 +
>  src/conf/domain_conf.c                             | 428 ++++++++++++---
>  src/conf/domain_conf.h                             |  10 +-
>  src/libvirt_private.syms                           |   6 +-
>  src/qemu/qemu_domain.c                             |   2 +-
>  src/qemu/qemu_process.c                            |  18 +-
>  src/util/virresctrl.c                              | 611 +++++++++++++++++++--
>  src/util/virresctrl.h                              |  55 +-
>  .../memorytune-colliding-allocs.xml                |  30 +
>  .../memorytune-colliding-cachetune.xml             |  32 ++
>  tests/genericxml2xmlindata/memorytune.xml          |  33 ++
>  tests/genericxml2xmltest.c                         |   5 +
>  .../linux-resctrl/resctrl/info/MB/bandwidth_gran   |   1 +
>  .../linux-resctrl/resctrl/info/MB/min_bandwidth    |   1 +
>  .../linux-resctrl/resctrl/info/MB/num_closids      |   1 +
>  tests/vircaps2xmldata/vircaps-x86_64-resctrl.xml   |   8 +
>  tests/virresctrldata/resctrl.schemata              |   1 +
>  21 files changed, 1280 insertions(+), 169 deletions(-)
>  create mode 100644 tests/genericxml2xmlindata/memorytune-colliding-allocs.xml
>  create mode 100644 tests/genericxml2xmlindata/memorytune-colliding-cachetune.xml
>  create mode 100644 tests/genericxml2xmlindata/memorytune.xml
>  create mode 100644 tests/vircaps2xmldata/linux-resctrl/resctrl/info/MB/bandwidth_gran
>  create mode 100644 tests/vircaps2xmldata/linux-resctrl/resctrl/info/MB/min_bandwidth
>  create mode 100644 tests/vircaps2xmldata/linux-resctrl/resctrl/info/MB/num_closids
> 

Reviewed-by: John Ferlan <jferlan@xxxxxxxxxx>
(series)

I'll push once the tree is open for 4.7.0 commits unless someone else
chimes in with other major issues that need to be addressed.

John

--
libvir-list mailing list
libvir-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvir-list



[Index of Archives]     [Virt Tools]     [Libvirt Users]     [Lib OS Info]     [Fedora Users]     [Fedora Desktop]     [Fedora SELinux]     [Big List of Linux Books]     [Yosemite News]     [KDE Users]     [Fedora Tools]

  Powered by Linux