Hi Barry, On Mon, Apr 27, 2015 at 1:22 PM, Barry Grussling <barry@xxxxxxxxxxxxx> wrote: > Hello Everyone, > > I am working on a device with a PLX PCIe switch and multiple > PCIe endpoints enumerated off an ARM RP. I am playing with > endpoint-to-endpoint communications. I have verified after > the PCIe bus is enumerated that I can send TLPs from > one endpoint to an enumerated BAR on another endpoint > and the TLP is directed to the other endpoint. I have further > verified that the RP receivies nothing as desired. I think this is what we've referred to as "peer-to-peer" traffic. Linux has no real infrastructure for supporting that, and there may be assumptions that devices don't do it, e.g., when configuring MPS or elsewhere. I don't *know* of problems; it's just mostly uncharted territory. > The issue I am looking for some advice on is how to tell > each endpoint what all the other endpoints BAR address is. > For the sake of argument, lets say I have N modules and > each module needs to know the enumerated endpoint > address of the other N-1 modules Bar0 address. > > The brute force/prototype method I have been testing is > scrape /proc/iomem after enumeration and write to each > card's registers via /dev/mem the address of all other > cards. This works, but is not something I would put > on my resume. I am now trying to "do it right". This > scraping also assumes bus addresses and host addresses > are the same. This seems to be true on my architecture. If you can get hold of struct pci_dev pointers for all the devices, you can use pci_resource_start() to get CPU physical addresses for the BARs. You can use pci_bus_address() to get the bus address. Bus addresses and CPU physical addresses are the same on some architectures, but certainly not all. A bus address is only meaningful to other devices under the same host bridge, but I'm guessing all the devices you're interested in are in the same hierarchy. > My preliminary driver design has a common base layer > that each endpoint's driver registers with when it is probed. > After the BAR addresses are assigned the endpoint driver > tells the common base layer its assigned address and > the base layer writes that into registers in all the > other registered endpoints via their respective drivers. All BARs should be assigned before a driver sees the device. After the PCI core calls the driver's probe function, the core should not modify anything the driver uses. > Before I go off boldly and and completely re-invent the > wheel I was wondering if something like this had > been contemplated or created yet, perhaps via > callback hooks or some other PCIe framework > thingy I am not aware of. Or is there another way > to achieve the same goal? > > Anyone have any feedback? > > Thanks for your time! > > Barry > -- > To unsubscribe from this list: send the line "unsubscribe linux-pci" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
