Hi, Sorry if I'm jumping too late. On Wed, Nov 14, 2018 at 03:49:16PM -0700, Keith Busch wrote: > Platforms may provide system memory where some physical address ranges > perform differently than others. These heterogeneous memory attributes are > common to the node that provides the memory and exported by the kernel. > > Add new documentation providing a brief overview of such systems and > the attributes the kernel makes available to aid applications wishing > to query this information. > > Signed-off-by: Keith Busch <keith.busch@xxxxxxxxx> > --- > Documentation/vm/numaperf.rst | 71 +++++++++++++++++++++++++++++++++++++++++++ As this document describes user-space interfaces it belongs to Documentation/admin-guide/mm. > 1 file changed, 71 insertions(+) > create mode 100644 Documentation/vm/numaperf.rst > > diff --git a/Documentation/vm/numaperf.rst b/Documentation/vm/numaperf.rst > new file mode 100644 > index 000000000000..5a3ecaff5474 > --- /dev/null > +++ b/Documentation/vm/numaperf.rst > @@ -0,0 +1,71 @@ > +.. _numaperf: > + > +================ > +NUMA Performance > +================ > + > +Some platforms may have multiple types of memory attached to a single > +CPU. These disparate memory ranges share some characteristics, such as > +CPU cache coherence, but may have different performance. For example, > +different media types and buses affect bandwidth and latency. > + > +A system supporting such heterogeneous memory groups each memory type > +under different "nodes" based on similar CPU locality and performance > +characteristics. Some memory may share the same node as a CPU, and > +others are provided as memory-only nodes. While memory only nodes do not > +provide CPUs, they may still be local to one or more compute nodes. The > +following diagram shows one such example of two compute noes with local > +memory and a memory only node for each of compute node: > + > + +------------------+ +------------------+ > + | Compute Node 0 +-----+ Compute Node 1 | > + | Local Node0 Mem | | Local Node1 Mem | > + +--------+---------+ +--------+---------+ > + | | > + +--------+---------+ +--------+---------+ > + | Slower Node2 Mem | | Slower Node3 Mem | > + +------------------+ +--------+---------+ > + > +A "memory initiator" is a node containing one or more devices such as > +CPUs or separate memory I/O devices that can initiate memory requests. A > +"memory target" is a node containing one or more CPU-accessible physical > +address ranges. > + > +When multiple memory initiators exist, accessing the same memory > +target may not perform the same as each other. The highest performing > +initiator to a given target is considered to be one of that target's > +local initiators. > + > +To aid applications matching memory targets with their initiators, > +the kernel provide symlinks to each other like the following example:: > + > + # ls -l /sys/devices/system/node/nodeX/initiator* > + /sys/devices/system/node/nodeX/targetY -> ../nodeY > + > + # ls -l /sys/devices/system/node/nodeY/target* > + /sys/devices/system/node/nodeY/initiatorX -> ../nodeX > + > +Applications may wish to consider which node they want their memory to > +be allocated from based on the nodes performance characteristics. If > +the system provides these attributes, the kernel exports them under the > +node sysfs hierarchy by appending the initiator_access directory under > +the node as follows:: > + > + /sys/devices/system/node/nodeY/initiator_access/ > + > +The kernel does not provide performance attributes for non-local memory > +initiators. The performance characteristics the kernel provides for > +the local initiators are exported are as follows:: > + > + # tree /sys/devices/system/node/nodeY/initiator_access > + /sys/devices/system/node/nodeY/initiator_access > + |-- read_bandwidth > + |-- read_latency > + |-- write_bandwidth > + `-- write_latency > + > +The bandwidth attributes are provided in MiB/second. > + > +The latency attributes are provided in nanoseconds. > + > +See also: https://www.uefi.org/sites/default/files/resources/ACPI_6_2.pdf > -- > 2.14.4 > -- Sincerely yours, Mike.
