Add documentation for core and hardware specific infiniband interfaces. The descriptions have been collected from git commit logs, reading through code and data sheets. Some drivers have incomplete doc and are annotated with the comment '[to be documented]'. Signed-off-by: Aishwarya Pant <aishpant@xxxxxxxxx> Reviewed-by: Hal Rosenstock <hal@xxxxxxxxxxxx> --- Changes in v4: - Typo GODs -> GIDs v3: - outbound -> inbound in description of port_rcv_constraint_errors v2: - Move infiniband interface from testing to stable - Fix typos - Update description of cap_mask, port_xmit_constraint_errors and port_rcv_constraint_errors - Add doc for hw_counters - Remove old documentation Documentation/ABI/stable/sysfs-class-infiniband | 818 +++++++++++++++++++++++ Documentation/ABI/testing/sysfs-class-infiniband | 16 - Documentation/infiniband/sysfs.txt | 129 +--- 3 files changed, 820 insertions(+), 143 deletions(-) create mode 100644 Documentation/ABI/stable/sysfs-class-infiniband delete mode 100644 Documentation/ABI/testing/sysfs-class-infiniband diff --git a/Documentation/ABI/stable/sysfs-class-infiniband b/Documentation/ABI/stable/sysfs-class-infiniband new file mode 100644 index 000000000000..f3acf3713a91 --- /dev/null +++ b/Documentation/ABI/stable/sysfs-class-infiniband @@ -0,0 +1,818 @@ +sysfs interface common for all infiniband devices +------------------------------------------------- + +What: /sys/class/infiniband/<device>/node_type +What: /sys/class/infiniband/<device>/node_guid +What: /sys/class/infiniband/<device>/sys_image_guid +Date: Apr, 2005 +KernelVersion: v2.6.12 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + node_type: (RO) Node type (CA, RNIC, usNIC, usNIC UDP, + switch or router) + + node_guid: (RO) Node GUID + + sys_image_guid: (RO) System image GUID + + +What: /sys/class/infiniband/<device>/node_desc +Date: Feb, 2006 +KernelVersion: v2.6.17 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + (RW) Update the node description with information such as the + node's hostname, so that IB network management software can tie + its view to the real world. + + +What: /sys/class/infiniband/<device>/fw_ver +Date: Jun, 2016 +KernelVersion: v4.10 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + (RO) Display firmware version + + +What: /sys/class/infiniband/<device>/ports/<port-num>/lid +What: /sys/class/infiniband/<device>/ports/<port-num>/rate +What: /sys/class/infiniband/<device>/ports/<port-num>/lid_mask_count +What: /sys/class/infiniband/<device>/ports/<port-num>/sm_sl +What: /sys/class/infiniband/<device>/ports/<port-num>/sm_lid +What: /sys/class/infiniband/<device>/ports/<port-num>/state +What: /sys/class/infiniband/<device>/ports/<port-num>/phys_state +What: /sys/class/infiniband/<device>/ports/<port-num>/cap_mask +Date: Apr, 2005 +KernelVersion: v2.6.12 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + + lid: (RO) Port LID + + rate: (RO) Port data rate (active width * active + speed) + + lid_mask_count: (RO) Port LID mask count + + sm_sl: (RO) Subnet manager SL for port's subnet + + sm_lid: (RO) Subnet manager LID for port's subnet + + state: (RO) Port state (DOWN, INIT, ARMED, ACTIVE or + ACTIVE_DEFER) + + phys_state: (RO) Port physical state (Sleep, Polling, + LinkUp, etc) + + cap_mask: (RO) Port capability mask. 2 bits here are + settable- IsCommunicationManagementSupported + (set when CM module is loaded) and IsSM (set via + open of issmN file). + + +What: /sys/class/infiniband/<device>/ports/<port-num>/link_layer +Date: Oct, 2010 +KernelVersion: v2.6.37 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + (RO) Link layer type information (Infiniband or Ethernet type) + + +What: /sys/class/infiniband/<device>/ports/<port-num>/counters/symbol_error +What: /sys/class/infiniband/<device>/ports/<port-num>/counters/port_rcv_errors +What: /sys/class/infiniband/<device>/ports/<port-num>/counters/port_rcv_remote_physical_errors +What: /sys/class/infiniband/<device>/ports/<port-num>/counters/port_rcv_switch_relay_errors +What: /sys/class/infiniband/<device>/ports/<port-num>/counters/link_error_recovery +What: /sys/class/infiniband/<device>/ports/<port-num>/counters/port_xmit_constraint_errors +What: /sys/class/infiniband/<device>/ports/<port-num>/counters/port_rcv_contraint_errors +What: /sys/class/infiniband/<device>/ports/<port-num>/counters/local_link_integrity_errors +What: /sys/class/infiniband/<device>/ports/<port-num>/counters/excessive_buffer_overrun_errors +What: /sys/class/infiniband/<device>/ports/<port-num>/counters/port_xmit_data +What: /sys/class/infiniband/<device>/ports/<port-num>/counters/port_rcv_data +What: /sys/class/infiniband/<device>/ports/<port-num>/counters/port_xmit_packets +What: /sys/class/infiniband/<device>/ports/<port-num>/counters/port_rcv_packets +What: /sys/class/infiniband/<device>/ports/<port-num>/counters/unicast_rcv_packets +What: /sys/class/infiniband/<device>/ports/<port-num>/counters/unicast_xmit_packets +What: /sys/class/infiniband/<device>/ports/<port-num>/counters/multicast_rcv_packets +What: /sys/class/infiniband/<device>/ports/<port-num>/counters/multicast_xmit_packets +What: /sys/class/infiniband/<device>/ports/<port-num>/counters/link_downed +What: /sys/class/infiniband/<device>/ports/<port-num>/counters/port_xmit_discards +What: /sys/class/infiniband/<device>/ports/<port-num>/counters/VL15_dropped +What: /sys/class/infiniband/<device>/ports/<port-num>/counters/port_xmit_wait +Date: Apr, 2005 +KernelVersion: v2.6.12 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + Errors info: + ----------- + + symbol_error: (RO) Total number of minor link errors detected on + one or more physical lanes. + + port_rcv_errors : (RO) Total number of packets containing an + error that were received on the port. + + port_rcv_remote_physical_errors : (RO) Total number of packets + marked with the EBP delimiter received on the port. + + port_rcv_switch_relay_errors : (RO) Total number of packets + received on the port that were discarded because they could not + be forwarded by the switch relay. + + link_error_recovery: (RO) Total number of times the Port + Training state machine has successfully completed the link error + recovery process. + + port_xmit_constraint_errors: (RO) Total number of packets not + transmitted from the switch physical port due to outbound raw + filtering or failing outbound partition or IP version check. + + port_rcv_constraint_errors: (RO) Total number of packets + received on the switch physical port that are discarded due to + inbound raw filtering or failing inbound partition or IP version + check. + + local_link_integrity_errors: (RO) The number of times that the + count of local physical errors exceeded the threshold specified + by LocalPhyErrors + + excessive_buffer_overrun_errors: (RO) This counter, indicates an + input buffer overrun. It indicates possible misconfiguration of + a port, either by the Subnet Manager (SM) or by user + intervention. It can also indicate hardware issues or extremely + poor link signal integrity + + Data info: + --------- + + port_xmit_data: (RO) Total number of data octets, divided by 4 + (lanes), transmitted on all VLs. This is 64 bit counter + + port_rcv_data: (RO) Total number of data octets, divided by 4 + (lanes), received on all VLs. This is 64 bit counter. + + port_xmit_packets: (RO) Total number of packets transmitted on + all VLs from this port. This may include packets with errors. + This is 64 bit counter. + + port_rcv_packets: (RO) Total number of packets (this may include + packets containing Errors. This is 64 bit counter. + + link_downed: (RO) Total number of times the Port Training state + machine has failed the link error recovery process and downed + the link. + + unicast_rcv_packets: (RO) Total number of unicast packets, + including unicast packets containing errors. + + unicast_xmit_packets: (RO) Total number of unicast packets + transmitted on all VLs from the port. This may include unicast + packets with errors. + + multicast_rcv_packets: (RO) Total number of multicast packets, + including multicast packets containing errors. + + multicast_xmit_packets: (RO) Total number of multicast packets + transmitted on all VLs from the port. This may include multicast + packets with errors. + + Misc info: + --------- + + port_xmit_discards: (RO) Total number of outbound packets + discarded by the port because the port is down or congested. + + VL15_dropped: (RO) Number of incoming VL15 packets dropped due + to resource limitations (e.g., lack of buffers) of the port. + + port_xmit_wait: (RO) The number of ticks during which the port + had data to transmit but no data was sent during the entire tick + (either because of insufficient credits or because of lack of + arbitration). + + Each of these files contains the corresponding value from the + port's Performance Management PortCounters attribute, as + described in the InfiniBand Architecture Specification. + + +What: /sys/class/infiniband/<device-name>/hw_counters/lifespan +What: /sys/class/infiniband/<device-name>/ports/<port-num>/hw_counters/lifespan +Date: May, 2016 +KernelVersion: 4.6 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + The optional "hw_counters" subdirectory can be under either the + parent device or the port subdirectories or both. If present, + there are a list of counters provided by the hardware. They may + match some of the counters in the counters directory, but they + often include many other counters. In addition to the various + counters, there will be a file named "lifespan" that configures + how frequently the core should update the counters when they are + being accessed (counters are not updated if they are not being + accessed). The lifespan is in milliseconds and defaults to 10 + unless set to something else by the driver. Users may echo a + value between 0-10000 to the lifespan file to set the length + of time between updates in milliseconds. + + +What: /sys/class/infiniband/<hca>/ports/<port-number>/gid_attrs/ndevs/<gid-index> +Date: November 29, 2015 +KernelVersion: 4.4.0 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: The net-device's name associated with the GID resides + at index <gid-index>. + +What: /sys/class/infiniband/<hca>/ports/<port-number>/gid_attrs/types/<gid-index> +Date: November 29, 2015 +KernelVersion: 4.4.0 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: The RoCE type of the associated GID resides at index <gid-index>. + This could either be "IB/RoCE v1" for IB and RoCE v1 based GODs + or "RoCE v2" for RoCE v2 based GIDs. + + +What: /sys/class/infiniband_mad/umadN/ibdev +What: /sys/class/infiniband_mad/umadN/port +What: /sys/class/infiniband_mad/issmN/ibdev +What: /sys/class/infiniband_mad/issmN/port +Date: Apr, 2005 +KernelVersion: v2.6.12 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + Each port of each InfiniBand device has a "umad" device and an + "issm" device attached. For example, a two-port HCA will have + two umad devices and two issm devices, while a switch will have + one device of each type (for switch port 0). + + ibdev: (RO) Show Infiniband (IB) device name + + port: (RO) Display port number + + +What: /sys/class/infiniband_mad/abi_version +Date: Apr, 2005 +KernelVersion: v2.6.12 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + (RO) Value is incremented if any changes are made that break + userspace ABI compatibility of umad & issm devices. + + +What: /sys/class/infiniband_cm/ucmN/ibdev +Date: Oct, 2005 +KernelVersion: v2.6.14 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + (RO) Display Infiniband (IB) device name + + +What: /sys/class/infiniband_cm/abi_version +Date: Oct, 2005 +KernelVersion: v2.6.14 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + (RO) Value is incremented if any changes are made that break + userspace ABI compatibility of ucm devices. + + +What: /sys/class/infiniband_verbs/uverbsN/ibdev +What: /sys/class/infiniband_verbs/uverbsN/abi_version +Date: Sept, 2005 +KernelVersion: v2.6.14 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + ibdev: (RO) Display Infiniband (IB) device name + + abi_version: (RO) Show ABI version of IB device specific + interfaces. + + +What: /sys/class/infiniband_verbs/abi_version +Date: Sep, 2005 +KernelVersion: v2.6.14 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + (RO) Value is incremented if any changes are made that break + userspace ABI compatibility of uverbs devices. + + +sysfs interface for Mellanox IB HCA low-level driver (mthca) +------------------------------------------------------------ + +What: /sys/class/infiniband/mthcaX/hw_rev +What: /sys/class/infiniband/mthcaX/hca_type +What: /sys/class/infiniband/mthcaX/board_id +Date: Apr, 2005 +KernelVersion: v2.6.12 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + hw_rev: (RO) Hardware revision number + + hca_type: (RO) Host Channel Adapter type: MT23108, MT25208 + (MT23108 compat mode), MT25208 or MT25204 + + board_id: (RO) Manufacturing board ID + + +sysfs interface for Chelsio T3 RDMA Driver (cxgb3) +-------------------------------------------------- + +What: /sys/class/infiniband/cxgb3_X/hw_rev +What: /sys/class/infiniband/cxgb3_X/hca_type +What: /sys/class/infiniband/cxgb3_X/board_id +Date: Feb, 2007 +KernelVersion: v2.6.21 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + hw_rev: (RO) Hardware revision number + + hca_type: (RO) HCA type. Here it is a driver short name. + It should normally match the name in its bus + driver structure (e.g. pci_driver::name). + + board_id: (RO) Manufacturing board id + + +sysfs interface for Mellanox ConnectX HCA IB driver (mlx4) +---------------------------------------------------------- + +What: /sys/class/infiniband/mlx4_X/hw_rev +What: /sys/class/infiniband/mlx4_X/hca_type +What: /sys/class/infiniband/mlx4_X/board_id +Date: Sep, 2007 +KernelVersion: v2.6.24 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + hw_rev: (RO) Hardware revision number + + hca_type: (RO) Host channel adapter type + + board_id: (RO) Manufacturing board ID + + +What: /sys/class/infiniband/mlx4_X/iov/ports/<port-num>/gids/<n> +What: /sys/class/infiniband/mlx4_X/iov/ports/<port-num>/admin_guids/<n> +What: /sys/class/infiniband/mlx4_X/iov/ports/<port-num>/pkeys/<n> +What: /sys/class/infiniband/mlx4_X/iov/ports/<port-num>/mcgs/ +What: /sys/class/infiniband/mlx4_X/iov/ports/<pci-slot-num>/ports/<m>/gid_idx/0 +What: /sys/class/infiniband/mlx4_X/iov/ports/<pci-slot-num>/ports/<m>/pkey_idx/<n> +Date: Aug, 2012 +KernelVersion: v3.6.15 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + The sysfs iov directory is used to manage and examine the port + P_Key and guid paravirtualization. This directory is added only + for the master -- slaves do not have it. + + Under iov/ports, the administrator may examine the gid and P_Key + tables as they are present in the device (and as are seen in the + "network view" presented to the SM). + + The "pkeys" and "gids" subdirectories contain one file for each + entry in the port's P_Key or GID table respectively. For + example, ports/1/pkeys/10 contains the value at index 10 in port + 1's P_Key table. + + gids/<n>: (RO) The physical port gids n = 0..127 + + admin_guids/<n>: (RW) Allows examining or changing the + administrative state of a given GUID + n = 0..127 + + pkeys/<n>: (RO) Displays the contents of the physical + key table n = 0..126 + + mcgs/: (RO) Muticast group table + + <m>/gid_idx/0: (RO) Display the GID mapping m = 1..2 + + <m>/pkey_idx/<n>: (RW) Writable except for RoCE pkeys. + m = 1..2, n = 0..126 + + Under the iov/<pci slot number> + directories, the admin may map the index + numbers in the physical tables (as under + iov/ports) to the paravirtualized index + numbers that guests see. + + For example, if the administrator, for + port 1 on guest 2 maps physical pkey + index 10 to virtual index 1, then that + guest, whenever it uses its pkey index + 1, will actually be using the real pkey + index 10. + + +What: /sys/class/infiniband/mlx4_X/iov/<pci-slot-num>/ports/<m>/smi_enabled +What: /sys/class/infiniband/mlx4_X/iov/<pci-slot-num>/ports/<m>/enable_smi_admin +Date: May, 2014 +KernelVersion: v3.15.7 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + Enabling QP0 on VFs for selected VF/port. By default, no VFs are + enabled for QP0 operation. + + smi_enabled: (RO) Indicates whether smi is currently enabled + for the indicated VF/port + + enable_smi_admin:(RW) Used by the admin to request that smi + capability be enabled or disabled for the + indicated VF/port. 0 = disable, 1 = enable. + + The requested enablement will occur at the next reset of the VF + (e.g. driver restart on the VM which owns the VF). + + +sysfs interface for NetEffect RNIC Low-Level iWARP driver (nes) +--------------------------------------------------------------- + +What: /sys/class/infiniband/nesX/hw_rev +What: /sys/class/infiniband/nesX/hca_type +What: /sys/class/infiniband/nesX/board_id +Date: Feb, 2008 +KernelVersion: v2.6.25 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + hw_rev: (RO) Hardware revision number + + hca_type: (RO) Host Channel Adapter type (NEX020) + + board_id: (RO) Manufacturing board id + + +sysfs interface for Chelsio T4/T5 RDMA driver (cxgb4) +----------------------------------------------------- + +What: /sys/class/infiniband/cxgb4_X/hw_rev +What: /sys/class/infiniband/cxgb4_X/hca_type +What: /sys/class/infiniband/cxgb4_X/board_id +Date: Apr, 2010 +KernelVersion: v2.6.35 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + + hw_rev: (RO) Hardware revision number + + hca_type: (RO) Driver short name. Should normally match + the name in its bus driver structure (e.g. + pci_driver::name) + + board_id: (RO) Manufacturing board id. (Vendor + device + information) + + +sysfs interface for Intel IB driver qib +--------------------------------------- + +What: /sys/class/infiniband/qibX/version +What: /sys/class/infiniband/qibX/hw_rev +What: /sys/class/infiniband/qibX/hca_type +What: /sys/class/infiniband/qibX/board_id +What: /sys/class/infiniband/qibX/boardversion +What: /sys/class/infiniband/qibX/nctxts +What: /sys/class/infiniband/qibX/localbus_info +What: /sys/class/infiniband/qibX/tempsense +What: /sys/class/infiniband/qibX/serial +What: /sys/class/infiniband/qibX/nfreectxts +What: /sys/class/infiniband/qibX/chip_reset +Date: May, 2010 +KernelVersion: v2.6.35 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + version: (RO) Display version information of installed software + and drivers. + + hw_rev: (RO) Hardware revision number + + hca_type: (RO) Host channel adapter type + + board_id: (RO) Manufacturing board id + + boardversion: (RO) Current version of the chip architecture + + nctxts: (RO) Return the number of user ports (contexts) + available + + localbus_info: (RO) Human readable localbus info + + tempsense: (RO) Display temp sense registers in decimal + + serial: (RO) Serial number of the HCA + + nfreectxts: (RO) The number of free user ports (contexts) + available. + + chip_reset: (WO) Reset the chip if possible by writing + "reset" to this file. Only allowed if no user + contexts are open that use chip resources. + + +What: /sys/class/infiniband/qibX/ports/N/sl2vl/[0-15] +Date: May, 2010 +KernelVersion: v2.6.35 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + (RO) The directory contains 16 files numbered 0-15 that specify + the Service Level (SL). Listing the SL files returns the Virtual + Lane (VL) as programmed by the SL. + +What: /sys/class/infiniband/qibX/ports/N/CCMgtA/cc_settings_bin +What: /sys/class/infiniband/qibX/ports/N/CCMgtA/cc_table_bin +Date: May, 2010 +KernelVersion: v2.6.35 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + Per-port congestion control. Both are binary attributes. + + cc_table_bin: (RO) Congestion control table size followed by + table entries. + + cc_settings_bin:(RO) Congestion settings: port control, control + map and an array of 16 entries for the + congestion entries - increase, timer, event log + trigger threshold and the minimum injection rate + delay. + +What: /sys/class/infiniband/qibX/ports/N/linkstate/loopback +What: /sys/class/infiniband/qibX/ports/N/linkstate/led_override +What: /sys/class/infiniband/qibX/ports/N/linkstate/hrtbt_enable +What: /sys/class/infiniband/qibX/ports/N/linkstate/status +What: /sys/class/infiniband/qibX/ports/N/linkstate/status_str +Date: May, 2010 +KernelVersion: v2.6.35 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + [to be documented] + + loopback: (WO) + led_override: (WO) + hrtbt_enable: (RW) + status: (RO) + + status_str: (RO) Displays information about the link state, + possible cable/switch problems, and hardware + errors. Possible states are- "Initted", + "Present", "IB_link_up", "IB_configured" or + "Fatal_Hardware_Error". + +What: /sys/class/infiniband/qibX/ports/N/diag_counters/rc_resends +What: /sys/class/infiniband/qibX/ports/N/diag_counters/seq_naks +What: /sys/class/infiniband/qibX/ports/N/diag_counters/rdma_seq +What: /sys/class/infiniband/qibX/ports/N/diag_counters/rnr_naks +What: /sys/class/infiniband/qibX/ports/N/diag_counters/other_naks +What: /sys/class/infiniband/qibX/ports/N/diag_counters/rc_timeouts +What: /sys/class/infiniband/qibX/ports/N/diag_counters/look_pkts +What: /sys/class/infiniband/qibX/ports/N/diag_counters/pkt_drops +What: /sys/class/infiniband/qibX/ports/N/diag_counters/dma_wait +What: /sys/class/infiniband/qibX/ports/N/diag_counters/unaligned +Date: May, 2010 +KernelVersion: v2.6.35 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + [to be documented] + + +sysfs interface for Mellanox Connect-IB HCA driver mlx5 +------------------------------------------------------- + +What: /sys/class/infiniband/mlx5_X/hw_rev +What: /sys/class/infiniband/mlx5_X/hca_type +What: /sys/class/infiniband/mlx5_X/reg_pages +What: /sys/class/infiniband/mlx5_X/fw_pages +Date: Jul, 2013 +KernelVersion: v3.11 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + [to be documented] + + +sysfs interface for Cisco VIC (usNIC) Verbs Driver +-------------------------------------------------- + +What: /sys/class/infiniband/usnic_X/board_id +What: /sys/class/infiniband/usnic_X/config +What: /sys/class/infiniband/usnic_X/qp_per_vf +What: /sys/class/infiniband/usnic_X/max_vf +What: /sys/class/infiniband/usnic_X/cq_per_vf +What: /sys/class/infiniband/usnic_X/iface +Date: Sep, 2013 +KernelVersion: v3.14 +Contact: Christian Benvenuti <benve@xxxxxxxxx>, + Dave Goodell <dgoodell@xxxxxxxxx>, + linux-rdma@xxxxxxxxxxxxxxx +Description: + + board_id: (RO) Manufacturing board id + + config: (RO) Report the configuration for this PF + + qp_per_vf: (RO) Queue pairs per virtual function. + + max_vf: (RO) Max virtual functions + + cq_per_vf: (RO) Completion queue per virtual function + + iface: (RO) Shows which network interface this usNIC + entry is associated to (visible with ifconfig). + +What: /sys/class/infiniband/usnic_X/qpn/summary +What: /sys/class/infiniband/usnic_X/qpn/context +Date: Sep, 2013 +KernelVersion: v3.14 +Contact: Christian Benvenuti <benve@xxxxxxxxx>, + Dave Goodell <dgoodell@xxxxxxxxx>, + linux-rdma@xxxxxxxxxxxxxxx +Description: + [to be documented] + + +sysfs interface for Emulex RoCE HCA Driver +------------------------------------------ + +What: /sys/class/infiniband/ocrdmaX/hw_rev +Date: Feb, 2014 +KernelVersion: v3.14 +Description: + hw_rev: (RO) Hardware revision number + +What: /sys/class/infiniband/ocrdmaX/hca_type +Date: Jun, 2014 +KernelVersion: v3.16 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + hca_type: (RO) Display FW version + + +sysfs interface for Intel Omni-Path driver (HFI1) +------------------------------------------------- + +What: /sys/class/infiniband/hfi1_X/hw_rev +What: /sys/class/infiniband/hfi1_X/board_id +What: /sys/class/infiniband/hfi1_X/nctxts +What: /sys/class/infiniband/hfi1_X/serial +What: /sys/class/infiniband/hfi1_X/chip_reset +What: /sys/class/infiniband/hfi1_X/boardversion +What: /sys/class/infiniband/hfi1_X/nfreectxts +What: /sys/class/infiniband/hfi1_X/tempsense +Date: May, 2016 +KernelVersion: v4.6 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + hw_rev: (RO) Hardware revision number + + board_id: (RO) Manufacturing board id + + nctxts: (RO) Total contexts available. + + serial: (RO) Board serial number + + chip_reset: (WO) Write "reset" to this file to reset the + chip if possible. Only allowed if no user + contexts are open that use chip resources. + + boardversion: (RO) Human readable board info + + nfreectxts: (RO) The number of free user ports (contexts) + available. + + tempsense: (RO) Thermal sense information + + +What: /sys/class/infiniband/hfi1_X/ports/N/CCMgtA/cc_settings_bin +What: /sys/class/infiniband/hfi1_X/ports/N/CCMgtA/cc_table_bin +What: /sys/class/infiniband/hfi1_X/ports/N/CCMgtA/cc_prescan +Date: May, 2016 +KernelVersion: v4.6 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + Per-port congestion control. + + cc_table_bin: (RO) CCA tables used by PSM2 Congestion control + table size followed by table entries. Binary + attribute. + + cc_settings_bin:(RO) Congestion settings: port control, control + map and an array of 16 entries for the + congestion entries - increase, timer, event log + trigger threshold and the minimum injection rate + delay. Binary attribute. + + cc_prescan: (RW) enable prescanning for faster BECN + response. Write "on" to enable and "off" to + disable. + +What: /sys/class/infiniband/hfi1_X/ports/N/sc2vl/[0-31] +What: /sys/class/infiniband/hfi1_X/ports/N/sl2sc/[0-31] +What: /sys/class/infiniband/hfi1_X/ports/N/vl2mtu/[0-15] +Date: May, 2016 +KernelVersion: v4.6 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + sc2vl/: (RO) 32 files (0 - 31) used to translate sl->vl + + sl2sc/: (RO) 32 files (0 - 31) used to translate sl->sc + + vl2mtu/: (RO) 16 files (0 - 15) used to determine MTU for vl + + +What: /sys/class/infiniband/hfi1_X/sdma_N/cpu_list +What: /sys/class/infiniband/hfi1_X/sdma_N/vl +Date: Sept, 2016 +KernelVersion: v4.8 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + sdma<N>/ contains one directory per sdma engine (0 - 15) + + cpu_list: (RW) List of cpus for user-process to sdma + engine assignment. + + vl: (RO) Displays the virtual lane (vl) the sdma + engine maps to. + + This interface gives the user control on the affinity settings + for the device. As an example, to set an sdma engine irq + affinity and thread affinity of a user processes to use the + sdma engine, which is "near" in terms of NUMA configuration, or + physical cpu location, the user will do: + + echo "3" > /proc/irq/<N>/smp_affinity_list + echo "4-7" > /sys/devices/.../sdma3/cpu_list + cat /sys/devices/.../sdma3/vl + 0 + echo "8" > /proc/irq/<M>/smp_affinity_list + echo "9-12" > /sys/devices/.../sdma4/cpu_list + cat /sys/devices/.../sdma4/vl + 1 + + to make sure that when a process runs on cpus 4,5,6, or 7, and + uses vl=0, then sdma engine 3 is selected by the driver, and + also the interrupt of the sdma engine 3 is steered to cpu 3. + Similarly, when a process runs on cpus 9,10,11, or 12 and sets + vl=1, then engine 4 will be selected and the irq of the sdma + engine 4 is steered to cpu 8. This assumes that in the above N + is the irq number of "sdma3", and M is irq number of "sdma4" in + the /proc/interrupts file. + + +sysfs interface for Intel(R) X722 iWARP i40iw driver +---------------------------------------------------- + +What: /sys/class/infiniband/i40iwX/hw_rev +What: /sys/class/infiniband/i40iwX/hca_type +What: /sys/class/infiniband/i40iwX/board_id +Date: Jan, 2016 +KernelVersion: v4.10 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + hw_rev: (RO) Hardware revision number + + hca_type: (RO) Show HCA type (I40IW) + + board_id: (RO) I40IW board ID + + +sysfs interface for QLogic qedr NIC Driver +------------------------------------------ + +What: /sys/class/infiniband/qedrX/hw_rev +What: /sys/class/infiniband/qedrX/hca_type +Date: Oct, 2016 +KernelVersion: v4.10 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + + hw_rev: (RO) Hardware revision number + + hca_type: (RO) Display HCA type + + +sysfs interface for VMware Paravirtual RDMA driver +-------------------------------------------------- + +What: /sys/class/infiniband/vmw_pvrdmaX/hw_rev +What: /sys/class/infiniband/vmw_pvrdmaX/hca_type +What: /sys/class/infiniband/vmw_pvrdmaX/board_id +Date: Oct, 2016 +KernelVersion: v4.10 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + + hw_rev: (RO) Hardware revision number + + hca_type: (RO) Host channel adapter type + + board_id: (RO) Display PVRDMA manufacturing board ID + + +sysfs interface for Broadcom NetXtreme-E RoCE driver +---------------------------------------------------- + +What: /sys/class/infiniband/bnxt_reX/hw_rev +What: /sys/class/infiniband/bnxt_reX/hca_type +Date: Feb, 2017 +KernelVersion: v4.11 +Contact: linux-rdma@xxxxxxxxxxxxxxx +Description: + hw_rev: (RO) Hardware revision number + + hca_type: (RO) Host channel adapter type diff --git a/Documentation/ABI/testing/sysfs-class-infiniband b/Documentation/ABI/testing/sysfs-class-infiniband deleted file mode 100644 index a86abe66a316..000000000000 --- a/Documentation/ABI/testing/sysfs-class-infiniband +++ /dev/null @@ -1,16 +0,0 @@ -What: /sys/class/infiniband/<hca>/ports/<port-number>/gid_attrs/ndevs/<gid-index> -Date: November 29, 2015 -KernelVersion: 4.4.0 -Contact: linux-rdma@xxxxxxxxxxxxxxx -Description: The net-device's name associated with the GID resides - at index <gid-index>. - -What: /sys/class/infiniband/<hca>/ports/<port-number>/gid_attrs/types/<gid-index> -Date: November 29, 2015 -KernelVersion: 4.4.0 -Contact: linux-rdma@xxxxxxxxxxxxxxx -Description: The RoCE type of the associated GID resides at index <gid-index>. - This could either be "IB/RoCE v1" for IB and RoCE v1 based GODs - or "RoCE v2" for RoCE v2 based GIDs. - - diff --git a/Documentation/infiniband/sysfs.txt b/Documentation/infiniband/sysfs.txt index 77570d16b170..9fab5062f84b 100644 --- a/Documentation/infiniband/sysfs.txt +++ b/Documentation/infiniband/sysfs.txt @@ -1,129 +1,4 @@ SYSFS FILES - For each InfiniBand device, the InfiniBand drivers create the - following files under /sys/class/infiniband/<device name>: - - node_type - Node type (CA, switch or router) - node_guid - Node GUID - sys_image_guid - System image GUID - - In addition, there is a "ports" subdirectory, with one subdirectory - for each port. For example, if mthca0 is a 2-port HCA, there will - be two directories: - - /sys/class/infiniband/mthca0/ports/1 - /sys/class/infiniband/mthca0/ports/2 - - (A switch will only have a single "0" subdirectory for switch port - 0; no subdirectory is created for normal switch ports) - - In each port subdirectory, the following files are created: - - cap_mask - Port capability mask - lid - Port LID - lid_mask_count - Port LID mask count - rate - Port data rate (active width * active speed) - sm_lid - Subnet manager LID for port's subnet - sm_sl - Subnet manager SL for port's subnet - state - Port state (DOWN, INIT, ARMED, ACTIVE or ACTIVE_DEFER) - phys_state - Port physical state (Sleep, Polling, LinkUp, etc) - - There is also a "counters" subdirectory, with files - - VL15_dropped - excessive_buffer_overrun_errors - link_downed - link_error_recovery - local_link_integrity_errors - port_rcv_constraint_errors - port_rcv_data - port_rcv_errors - port_rcv_packets - port_rcv_remote_physical_errors - port_rcv_switch_relay_errors - port_xmit_constraint_errors - port_xmit_data - port_xmit_discards - port_xmit_packets - symbol_error - - Each of these files contains the corresponding value from the port's - Performance Management PortCounters attribute, as described in - section 16.1.3.5 of the InfiniBand Architecture Specification. - - The "pkeys" and "gids" subdirectories contain one file for each - entry in the port's P_Key or GID table respectively. For example, - ports/1/pkeys/10 contains the value at index 10 in port 1's P_Key - table. - - There is an optional "hw_counters" subdirectory that may be under either - the parent device or the port subdirectories or both. If present, - there are a list of counters provided by the hardware. They may match - some of the counters in the counters directory, but they often include - many other counters. In addition to the various counters, there will - be a file named "lifespan" that configures how frequently the core - should update the counters when they are being accessed (counters are - not updated if they are not being accessed). The lifespan is in milli- - seconds and defaults to 10 unless set to something else by the driver. - Users may echo a value between 0 - 10000 to the lifespan file to set - the length of time between updates in milliseconds. - -MTHCA - - The Mellanox HCA driver also creates the files: - - hw_rev - Hardware revision number - fw_ver - Firmware version - hca_type - HCA type: "MT23108", "MT25208 (MT23108 compat mode)", - or "MT25208" - -HFI1 - - The hfi1 driver also creates these additional files: - - hw_rev - hardware revision - board_id - manufacturing board id - tempsense - thermal sense information - serial - board serial number - nfreectxts - number of free user contexts - nctxts - number of allowed contexts (PSM2) - chip_reset - diagnostic (root only) - boardversion - board version - - sdma<N>/ - one directory per sdma engine (0 - 15) - sdma<N>/cpu_list - read-write, list of cpus for user-process to sdma - engine assignment. - sdma<N>/vl - read-only, vl the sdma engine maps to. - - The new interface will give the user control on the affinity settings - for the hfi1 device. - As an example, to set an sdma engine irq affinity and thread affinity - of a user processes to use the sdma engine, which is "near" in terms - of NUMA configuration, or physical cpu location, the user will do: - - echo "3" > /proc/irq/<N>/smp_affinity_list - echo "4-7" > /sys/devices/.../sdma3/cpu_list - cat /sys/devices/.../sdma3/vl - 0 - echo "8" > /proc/irq/<M>/smp_affinity_list - echo "9-12" > /sys/devices/.../sdma4/cpu_list - cat /sys/devices/.../sdma4/vl - 1 - - to make sure that when a process runs on cpus 4,5,6, or 7, - and uses vl=0, then sdma engine 3 is selected by the driver, - and also the interrupt of the sdma engine 3 is steered to cpu 3. - Similarly, when a process runs on cpus 9,10,11, or 12 and sets vl=1, - then engine 4 will be selected and the irq of the sdma engine 4 is - steered to cpu 8. - This assumes that in the above N is the irq number of "sdma3", - and M is irq number of "sdma4" in the /proc/interrupts file. - - ports/1/ - CCMgtA/ - cc_settings_bin - CCA tables used by PSM2 - cc_table_bin - cc_prescan - enable prescaning for faster BECN response - sc2v/ - 32 files (0 - 31) used to translate sl->vl - sl2sc/ - 32 files (0 - 31) used to translate sl->sc - vl2mtu/ - 16 (0 - 15) files used to determine MTU for vl +The sysfs interface has moved to +Documentation/ABI/stable/sysfs-class-infiniband. -- 2.16.1 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
