Hi,
On 12/17/21 13:35, Rob Herring wrote:
On Fri, Dec 17, 2021 at 1:08 PM Robin Murphy <robin.murphy@xxxxxxx> wrote:
On 2021-12-17 18:14, Rob Herring wrote:
On Fri, Dec 17, 2021 at 10:57 AM Robin Murphy <robin.murphy@xxxxxxx> wrote:
Hi Rob,
On 2021-12-16 23:31, Rob Herring wrote:
Use the minimum CPU h/w id of the CPUs associated with the cache for the
cache 'id'. This will provide a stable id value for a given system. As
we need to check all possible CPUs, we can't use the shared_cpu_map
which is just online CPUs. There's not a cache to CPUs mapping in DT, so
we have to walk all CPU nodes and then walk cache levels.
I believe another expected use of the cache ID exposed in sysfs is to
program steering tags for cache stashing (typically in VFIO-based
userspace drivers like DPDK so we can't realistically mediate it any
other way). There were plans afoot last year to ensure that ACPI PPTT
could provide the necessary ID values for arm64 systems which will
typically be fairly arbitrary (but unique) due to reflecting underlying
interconnect routing IDs. Assuming that there will eventually be some
interest in cache stashing on DT-based systems too, we probably want to
allow for an explicit ID property on DT cache nodes in a similar manner.
If you have a suggestion for ID values that correspond to the h/w,
then we can add them. I'd like a bit more than just trusting that ID
is something real.
While the ACPI folks may be willing to take an arbitrary index, it's
something we (mostly) avoid for DT.
Not really. On the CHI side there are two fields - StashNID, which could
be any node ID value depending on the interconnect layout, plus
(optionally) StashLPID to address a specific cache within that node if
it's something like a CPU cluster. However, how a PCIe TLP steering tag
translates to those fields in the resulting CHI flit is largely up to
the root complex.
Knowing next to nothing about CHI, this means pretty much nothing to me. :(
I would guess there is a bit more to supporting CHI in DT systems than
just a cache ID.
I think it's going to be more like a "reg" property than a nice
validatable index.
That said, I think it does make sense to have some kind of
auto-generated fallback scheme *as well*, since I'm sure there will be
plenty systems which care about MPAM but don't support stashing, and
therefore wouldn't have a meaningful set of IDs to populate their DT
with. Conversely I think that might also matter for ACPI too - one point
I remember from previous discussions is that PPTT may use a compact
representation where a single entry represents all equivalent caches at
that level, so I'm not sure we can necessarily rely on IDs out of that
path being unique either.
AIUI, cache ids break the compact representation.
Right, firmware authors can't use it if they do want to specify IDs, but
that also means that if we find we *are* consuming a compact PPTT, then
chances are we're not getting meaningful IDs out of it for MPAM to rely on.
Sounds like broken firmware is in our future. ;) Or ACPI can default
to the same id scheme.
Yah, that is a problem. The ID's provided by the ACPI cache ID field are
as officially meaningless as the ones we can generate from the existing
fw_token mechanism. Given that, they don't really add anything beyond
what we can achieve simply by encoding the level somewhere in the
fw_token currently in use if we want something that is globally unique
rather than just unique for a given cache level+I/D. Their one advantage
though is that they can be more human readable at the cost of 2-3X the
size of the table, with the additional problem of having to worry about
them being populated in all the cache structures in the table. Its
almost easier to revisit some of the earlier discussion and generate a
uniq id, and then renumber them at the end.
If you want to encode some kind of routing ID in them, then that will
have to be standardized, and I would guess it might be easier to add the
routing ID's to the structure than retroactively add meaning to the ID
field if anyone is actually using it. Or just create yet another lookup
table to translate the id to something meaningful.
