On Fri, 7 May 2021 15:51:14 -0700 Dan Williams <dan.j.williams@xxxxxxxxx> wrote: > Changes since v2 [1]: > - Add some rationale for moving definitions out of mem.c into mem.h > (Jonathan) > > - Fixup CXL_DEVICE_REGS kernel doc and declare the fixup for the > struct cxl_mem kernel-doc in the changelog (Jonathan) > > - Fixup cxl_setup_device_regs() kernel-doc (Jonathan) > > - Cleanup comment in cxl_root_alloc() (Jonathan) > > - [not changed] refactor device creation into alloc_and_add() > anti-pattern. > > - Add kernel doc to cxl_address_space_dev_add() to clarify why @host is > needed as an arg. (Jonathan) > > - Describe what the port devices are representing in "cxl/port: > Introduce cxl_port objects" (Jonathan) > > - Explain the rationale for /sys/bus/cxl (Bjorn) > > [1]: http://lore.kernel.org/r/161728744224.2474040.12854720917440712854.stgit@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx Hi Dan, What's the base for this series? Given it was nearly ready to go (as far as I'm concerned anyway), I thought I'd give it a spin but seems it doesn't have some changes from fixes that have gone upstream. Thanks, Jonathan > > --- > > Plans for v5.14: > > This series is a starting point for the persistent memory and dynamic > HDM decoder manipulation support targeting the v5.14 kernel. Among the > tasks to complete in that timeframe are: > > - Region creation including CXL label support > - LIBNVDIMM integration for surfacing /dev/pmemX and /dev/daxX.Y devices > on CXL resources > - HDM decoder enumeration and programming for setting up PMEM mappings > alongside any "System RAM" mappings established by platform firmware > - CDAT-DOE support in support of dynamically created NUMA nodes > - CXL PMEM Shutdown semantics (global persistence flush coordination) > > Contributions to cxl.git targeting the next merge window require a > non-author Reviewed-by. A patch with a Reviewed-by, no outstanding > objections, and a 3-5 day quiet period on the list is subject to be > applied to a non-rebasing cxl.git branch and merged into cxl.git/next. > Contributions targeting a current -rc (fixes) may go in on an expedited > schedule with only an Acked-by. > > --- > > CXL Port Topology: > > The enumeration starts with the ACPI0017 driver registering a 'struct > cxl_root' object to establish the top of a cxl_port topology. It then > scans the ACPI bus looking for ACPI0016 instances. The cxl_root object > is a singleton* anchor to hang "address-space" objects and be a parent > device for the downstream 'struct cxl_port' instances. An address-space > has a 1:1 relationship with a platform defined memory resource range, > like _CRS for PCIE Host Bridges. Use module parameters to model a > root-level HDM decoder that all downstream ports further decode, to be > replaced with a Code First ECN to do the same. > > Each address space is modeled as a sysfs object that also shows up in > /proc/iomem as "CXL Address Space". That iomem resource is functionally > equivalent to the root-level 'PCI Bus' resources for PCIE.mmio while > 'CXL Address Space' indicates space for CXL.mem to be mapped. "System > RAM" and "Persistent Memory", when mapped by HDM decoders, will appear > as child CXL.mem resources. > > Once a 'struct cxl_root' is established the host bridge is modeled as 1 > upstream 'struct cxl_port' and N downstream 'struct cxl_port' instances > (one per Root Port), just like a PCIE switch. The host-bridge upstream > port optionally has the HDM decoder registers from the CHBCR if the > host-bridge has multiple PCIE/CXL root ports. Single-ported host bridges > will not have HDM decoders in the CHBCR space (see CHBCR note in > 8.2.5.12 CXL HDM Decoder Capability Structure), but the 'struct > cxl_port' object is still needed to represent other CXL capabilities and > access port-specific component registers outside of HDM decoders. > > Each 'struct cxl_port' has a 'target_id' attribute that answers the > question "what port am I in my upstream port's HDM decoder target > list?". For the host-bridge struct cxl_port, the first tier of ports > below cxl_root.port, the id is derived from the ordinal mapping of the > ACPI0016 id (instance id, _UID, or other handle TBD), for all other > ports the id is the PCIE Root Port ID from the Link Capabilities > register [1]. The mapping of ordinal port identifiers relative to their > parent may change once libcxl and cxl-cli prove out region creation, or > a better option is found to establish a static device path / persistent > naming scheme. System software must not assume that 'struct cxl_port' > device names will be static from one boot to the next. > > See patch7 for a tree(1) topology picture of what QEMU is producing > today with this enabling. > > * cxl_root is singleton only by convention. A given cxl_root could > represent 1 to N address spaces, this patch set chooses to implement 1 > cxl_root for all address spaces. > > [1]: CXL 2.0 8.2.5.12.8 CXL HDM Decoder 0 Target List Low Register > (Offset 24h) ...The Target Port Identifier for a given Downstream Port > is reported via Port Number field in Link Capabilities Register. (See > PCI Express Base Specification). > > --- > > Dan Williams (8): > cxl/mem: Move some definitions to mem.h > cxl/mem: Introduce 'struct cxl_regs' for "composable" CXL devices > cxl/core: Rename bus.c to core.c > cxl/core: Refactor CXL register lookup for bridge reuse > cxl/acpi: Introduce ACPI0017 driver and cxl_root > cxl/Kconfig: Default drivers to CONFIG_CXL_BUS > cxl/port: Introduce cxl_port objects > cxl/acpi: Add module parameters to stand in for ACPI tables > > > Documentation/driver-api/cxl/memory-devices.rst | 6 > drivers/cxl/Kconfig | 16 + > drivers/cxl/Makefile | 6 > drivers/cxl/acpi.c | 215 +++++++++ > drivers/cxl/bus.c | 29 - > drivers/cxl/core.c | 561 +++++++++++++++++++++++ > drivers/cxl/cxl.h | 148 ++++-- > drivers/cxl/mem.c | 97 +--- > drivers/cxl/mem.h | 82 +++ > 9 files changed, 999 insertions(+), 161 deletions(-) > create mode 100644 drivers/cxl/acpi.c > delete mode 100644 drivers/cxl/bus.c > create mode 100644 drivers/cxl/core.c > create mode 100644 drivers/cxl/mem.h > > base-commit: a38fd8748464831584a19438cbb3082b5a2dab15
