> On May 27, 2015, at 10:39 AM, Mark Rutland <mark.rutland@xxxxxxx> wrote:
>
> On Tue, May 26, 2015 at 11:20:59PM +0100, Chalamarla, Tirumalesh wrote:
>> This is some thing we also like to see in ITS and SMMU drivers.
>>> On Mar 24, 2015, at 8:50 AM, Mark Rutland <mark.rutland@xxxxxxx> wrote:
>>>
>>> Hi all,
>>>
>>> For some devices, identification of particular masters is critical to
>>> their operation (e.g. IOMMUs, MSI controllers). The identity of masters
>>> is determined from sideband signals on the bus, and it may or may not be
>>> possible to probe and/or configure this information. Worse still, these
>>> information may be rewritten by intermediate buses, so the
>>> information for a given master may differ at different points in the bus
>>> hierarchy.
>>>
>>> We currently describe this master information for devices attached to
>>> IOMMUs, with a master ID being encoded in the iommu-cells. However this
>>> only covers the ID between the master and its IOMMU(s), and we don't
>>> currently have a mechanism to describe the master IDs as they are seen
>>> by devices beyond the IOMMU(s), or in the absence of any IOMMU(s).
>>>
>>> The following are example of master IDs:
>>> * PCIe Requester IDs (RIDs) (bus:device.function AKA BDF)
>>> * SMMU Stream IDs (SIDs)
>>> * GICv3 ITS Device IDs (DIDs)
>>>
>>> In the case of a system with PCIe, SMMU and GICv3, the master IDs are
>>> rewritten in a chain of RID=>SID=>DID, as in the following diagram:
>>>
>>> +-------------+
>>> | PCIe master |
>>> +-------------+
>>> ||
>>> || Requester ID (RID)
>>> || Probeable from RC.
>>> \/
>>> +-------------------+
>>> | PCIe Root Complex |
>>> +-------------------+
>>> ||
>>> || SMMU Stream ID (SID)
>>> || Derived from RID, static non-probeable mapping.
>>> \/
>>> +--------------+
>>> | SMMU (IOMMU) |
>>> +--------------+
>>> ||
>>> || ITS Device ID (DID)
>>> || Derived from SID, static non-probeable mapping.
>>> \/
>>> +----------------------------+
>>> | GICv3 ITS (MSI controller) |
>>> +----------------------------+
>>>
>>> In simpler cases, you might have a simpler set of master ID
>>> translations, e.g. just a DID:
>>>
>>> +-----------------+
>>> | Platform device |
>>> +-----------------+
>>> ||
>>> || ITS Device ID (DID)
>>> || Hard-wired on the bus.
>>> \/
>>> +----------------------------+
>>> | GICv3 ITS (MSI controller) |
>>> +----------------------------+
>>>
>>> Ignoring current bindings for the moment, I can see how we can describe
>>> this with a generic master id-binding, with master-id-map along the
>>> lines of interrupt-map, with a tuple format like:
>>> <child-id-base child-id-length parent parent-id-base>
>>>
>>> For the PCIe example, this would look something like the following (with
>>> properties omitted for brevity):
>>>
>>> PCI: pci@af000000 {
>>> ...
>>>
>>> /* Requester ID of PCIe endpoints, implicit at runtime */
>>> master-id-cells = <1>;
>>>
>>> /* RIDS idmapped to SIDS @ SMMU */
>>> master-id-map = <0 0x10000 &SMMU 0>;
>>> }
>>>
>>> SMMU: iommu@bf000000 {
>>> ...
>>>
>>> /* SID, derived from RID */
>>> master-id-cells = <1>;
>>>
>>> /*
>>> * For some reason the high bit of the ID was negated.
>>> */
>>> master-id-map = <0x0000 0x8000 &ITS 0x0 0x8000>,
>>> <0x8000 0x8000 &ITS 0x0 0x0000>;
>>>
>>> };
>>>
>>> ITS: its@cf000000 {
>>> ...
>>>
>>> /* DID, derived from SID */
>>> master-id-cells = <2>;
>>>
>>> /*
>>> * Master traffic not propagated beyond this point, so no
>>> * master-id-ranges
>>> */
>>> };
>>
>> I think it is nice to have max IDs supported by masters. so that drivers can check and enforce.
>
> The set of IDs that we expect to transform (i.e. those masters use)
> would be implicit in the first master-id-map from a master. In the PCI
> example above, no master is expected to generate an ID outside of the
> range 0-0x10000 inclusive (and that's all the SMMU would see).
>
> For devices which are not hotpluggable, the nodes for those devices
> would describe the specific set of IDs they use.
>
> Generally, between a master and a slave there might not be one
> contiguous set of valid IDs as these may be rewritten along the way (as
> happens at the SMMU between the PCI root complex and the ITS in the
> example above).
>
> Which drivers do you think need this information? What exactly are they
> trying to check and enforce?
>
I think, i miss understood by the names, so it is not base it is some thing like
x => f(x) => y (and f(x) is bitwise | )
ok now i see it.
>>> For the simpler case, this would look something like:
>>>
>>> DEV: device@af000000 {
>>> master-id-cells = <1>;
>>> master-ids = <0xf>, <0xb>;
>>> master-id-map = <0 0xf &ITS 0 0>;
>>> };
>>>
>>> ITS: its@cf000000 {
>>> ...
>>>
>>> /* DID */
>>> master-id-cells = <2>;
>>> };
>>>
>> Is this is not depending heavily on discover order, how do drivers
>> know which device to get which ID. is it implicitly assumed in
>> discovery order?
>
> I'm not sure I follow the question. Could you elaborate?
>
>> what happens to hot pluggable devices.
>
> It would be necessary to be able to discover the ID assigned to the
> device by the hotpluggable bus. For example, this could depend on which
> slot the device is plugged into.
>
> If you can't discover the ID associated with a hotpluggable device from
> the bus it is plugged into I can't see how hotplug could work.
>
> From that point on I would expect the ID transformations to be static as
> in the example.
>
what if i have an SMMU with two ITS masters and i want to decide which one to use based on runtime like which CPU i am serving.
i am thinking in terms go NUMA based systems, where my pci device is on node 0, if the interrupt is targeted to CPU on node 1, i would like to use ITS1, otherwise
ITS0.
is it possible to specify some thing like that or do you think its is best to leave to ITS driver.
Thanks,
Tirumalesh.
> Thanks,
> Mark.
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
