On Sun, Jan 20, 2019 at 8:20 AM Rafael J. Wysocki <rafael@xxxxxxxxxx> wrote: > > On Sat, Jan 19, 2019 at 5:56 PM Dan Williams <dan.j.williams@xxxxxxxxx> wrote: > > > > On Sat, Jan 19, 2019 at 1:01 AM Greg Kroah-Hartman > > <gregkh@xxxxxxxxxxxxxxxxxxx> wrote: > > > > > > On Fri, Jan 18, 2019 at 01:08:02PM -0800, Dan Williams wrote: > > > > On Thu, Jan 17, 2019 at 3:41 AM Rafael J. Wysocki <rafael@xxxxxxxxxx> wrote: > > > > > > > > > > On Wed, Jan 16, 2019 at 6:59 PM Keith Busch <keith.busch@xxxxxxxxx> wrote: > > > > > > > > > > > > Add entries for memory initiator and target node class attributes. > > > > > > > > > > > > Signed-off-by: Keith Busch <keith.busch@xxxxxxxxx> > > > > > > > > > > I would recommend combining this with the previous patch, as the way > > > > > it is now I need to look at two patches at the time. :-) > > > > > > > > > > > --- > > > > > > Documentation/ABI/stable/sysfs-devices-node | 25 ++++++++++++++++++++++++- > > > > > > 1 file changed, 24 insertions(+), 1 deletion(-) > > > > > > > > > > > > diff --git a/Documentation/ABI/stable/sysfs-devices-node b/Documentation/ABI/stable/sysfs-devices-node > > > > > > index 3e90e1f3bf0a..a9c47b4b0eee 100644 > > > > > > --- a/Documentation/ABI/stable/sysfs-devices-node > > > > > > +++ b/Documentation/ABI/stable/sysfs-devices-node > > > > > > @@ -90,4 +90,27 @@ Date: December 2009 > > > > > > Contact: Lee Schermerhorn <lee.schermerhorn@xxxxxx> > > > > > > Description: > > > > > > The node's huge page size control/query attributes. > > > > > > - See Documentation/admin-guide/mm/hugetlbpage.rst > > > > > > \ No newline at end of file > > > > > > + See Documentation/admin-guide/mm/hugetlbpage.rst > > > > > > + > > > > > > +What: /sys/devices/system/node/nodeX/classY/ > > > > > > +Date: December 2018 > > > > > > +Contact: Keith Busch <keith.busch@xxxxxxxxx> > > > > > > +Description: > > > > > > + The node's relationship to other nodes for access class "Y". > > > > > > + > > > > > > +What: /sys/devices/system/node/nodeX/classY/initiator_nodelist > > > > > > +Date: December 2018 > > > > > > +Contact: Keith Busch <keith.busch@xxxxxxxxx> > > > > > > +Description: > > > > > > + The node list of memory initiators that have class "Y" access > > > > > > + to this node's memory. CPUs and other memory initiators in > > > > > > + nodes not in the list accessing this node's memory may have > > > > > > + different performance. > > > > > > > > > > This does not follow the general "one value per file" rule of sysfs (I > > > > > know that there are other sysfs files with more than one value in > > > > > them, but it is better to follow this rule as long as that makes > > > > > sense). > > > > > > > > > > > + > > > > > > +What: /sys/devices/system/node/nodeX/classY/target_nodelist > > > > > > +Date: December 2018 > > > > > > +Contact: Keith Busch <keith.busch@xxxxxxxxx> > > > > > > +Description: > > > > > > + The node list of memory targets that this initiator node has > > > > > > + class "Y" access. Memory accesses from this node to nodes not > > > > > > + in this list may have differet performance. > > > > > > -- > > > > > > > > > > Same here. > > > > > > > > > > And if you follow the recommendation given in the previous message > > > > > (add "initiators" and "targets" subdirs under "classX"), you won't > > > > > even need the two files above. > > > > > > > > This recommendation is in conflict with Greg's feedback about kobject > > > > usage. If these are just "vanity" subdirs I think it's better to have > > > > a multi-value sysfs file. This "list" style is already commonplace for > > > > the /sys/devices/system hierarchy. > > > > > > If you do a subdirectory "correctly" (i.e. a name for an attribute > > > group), that's fine. Just do not ever create a kobject just for a > > > subdir, that will mess up userspace. > > > > > > And I hate the "multi-value" sysfs files, where at all possible, please > > > do not copy past bad mistakes there. If you can make them individual > > > files, please do that, it makes it easier to maintain and code the > > > kernel for. > > > > I agree in general about multi-value sysfs, but in this case we're > > talking about a mask. Masks are a single value. That said I can get on > > board with calling what 'cpulist' does a design mistake (human > > readable mask), but otherwise switching to one file per item in the > > mask is a mess for userspace to consume. > > Can you please refer to my response to Keith? Ah, ok I missed the patch4 comments and was reading this one in isolation... which also bolsters your comment about squashing these two patches together. > If you have "initiators" and "targets" under "classX" and a list of > symlinks in each of them, I don't see any kind of a mess here. In this instance, I think having symlinks at all is "messy" vs just having a mask. Yes, you're right, if we have the proposed symlinks from patch4 there is no need for these _nodelist attributes, and those symlinks would be better under "initiator" and "target" directories. However, I'm arguing going the other way, just have the 2 mask attributes and no symlinks. The HMAT erodes the concept of "numa nodes" typically being a single digit number space per platform. Given increasing numbers of memory target types and initiator devices its going to be cumbersome to have userspace walk multiple symlinks vs just reading a mask and opening the canonical path for a node directly. This is also part of the rationale for only emitting one "class" (initiator / target performance profile) by default. There's an N-to-N initiator-target description in the HMAT. When / if we decide emit more classes the more work userspace would need to do to convert directory structures back into data.
