On Tue, 15 Dec 2020 01:03:56 -0800 Saeed Mahameed wrote: > +PCI controllers > +--------------- > +In most cases a PCI device has only one controller. A controller consists of > +potentially multiple physical and virtual functions. Such PCI function consists > +of one or more ports. s/Such// you say consists in two consecutive sentences. > This port of the function is represented by the devlink eswitch port. "This port of the function"? Why not just "Each port"? > +A PCI Device connected to multiple CPUs or multiple PCI root complexes or Why is device capitalized all of the sudden? > +SmartNIC, however, may have multiple controllers. For a device with multiple a SmartNIC or SmartNICs > +controllers, each controller is distinguished by a unique controller number. > +An eswitch on the PCI device support ports of multiple controllers. eswitch is on a PCI device? > +An example view of a system with two controllers:: > + > + --------------------------------------------------------- > + | | > + | --------- --------- ------- ------- | > + ----------- | | vf(s) | | sf(s) | |vf(s)| |sf(s)| | > + | server | | ------- ----/---- ---/----- ------- ---/--- ---/--- | > + | pci rc |=== | pf0 |______/________/ | pf1 |___/_______/ | > + | connect | | ------- ------- | > + ----------- | | controller_num=1 (no eswitch) | > + ------|-------------------------------------------------- > + (internal wire) > + | > + --------------------------------------------------------- > + | devlink eswitch ports and reps | > + | ----------------------------------------------------- | > + | |ctrl-0 | ctrl-0 | ctrl-0 | ctrl-0 | ctrl-0 |ctrl-0 | | > + | |pf0 | pf0vfN | pf0sfN | pf1 | pf1vfN |pf1sfN | | > + | ----------------------------------------------------- | > + | |ctrl-1 | ctrl-1 | ctrl-1 | ctrl-1 | ctrl-1 |ctrl-1 | | > + | |pf0 | pf0vfN | pf0sfN | pf1 | pf1vfN |pf1sfN | | > + | ----------------------------------------------------- | > + | | > + | | > + ----------- | --------- --------- ------- ------- | > + | smartNIC| | | vf(s) | | sf(s) | |vf(s)| |sf(s)| | > + | pci rc |==| ------- ----/---- ---/----- ------- ---/--- ---/--- | > + | connect | | | pf0 |______/________/ | pf1 |___/_______/ | > + ----------- | ------- ------- | > + | | > + | local controller_num=0 (eswitch) | > + --------------------------------------------------------- > + > +In above example, external controller (identified by controller number = 1) > +doesn't have eswitch. Local controller (identified by controller number = 0) > +has the eswitch. Devlink instance on local controller has eswitch devlink > +ports representing ports for both the controllers. > + > +Port function configuration > +=========================== > + > +A user can configure the port function attribute before enumerating the s/A user/User/ /port function attribute/$something_meaningful/ > +PCI function. Usually it means, user should configure port function attribute attributes, plural > +before a bus specific device for the function is created. However, when > +SRIOV is enabled, virtual function devices are created on the PCI bus. > +Hence, function attribute should be configured before binding virtual > +function device to the driver. > + > +User may set the hardware address of the function represented by the devlink > +port function. For Ethernet port function this means a MAC address.
