On Fri, Jun 22, 2012 at 01:04:03PM +0200, Thierry Reding wrote:
> On Tue, Jun 19, 2012 at 11:31:39AM -1000, Mitch Bradley wrote:
> > On 6/19/2012 3:30 AM, Thierry Reding wrote:
> > >On Fri, Jun 15, 2012 at 08:12:36AM +0200, Thierry Reding wrote:
> > >>On Thu, Jun 14, 2012 at 01:50:56PM -0600, Stephen Warren wrote:
> > >>>On 06/14/2012 01:29 PM, Thierry Reding wrote:
> > >>>>On Thu, Jun 14, 2012 at 12:30:50PM -0600, Stephen Warren wrote:
> > >>>>>On 06/14/2012 03:19 AM, Thierry Reding wrote:
> > >>>...
> > >>>>>>#address-cells = <1>; #size-cells = <1>;
> > >>>>>>
> > >>>>>>pci@80000000 {
> > >>>>>
> > >>>>>I'm still not convinced that using the address of the port's
> > >>>>>registers is the correct way to represent each port. The port
> > >>>>>index seems much more useful.
> > >>>>>
> > >>>>>The main reason here is that there are a lot of registers that
> > >>>>>contain fields for each port - far more than the combination of
> > >>>>>this node's reg and ctrl-offset (which I assume is an address
> > >>>>>offset for just one example of this issue) properties can
> > >>>>>describe. The bit position and bit stride of these fields isn't
> > >>>>>necessarily the same in each register. Do we want a property like
> > >>>>>ctrl-offset for every single type of field in every single shared
> > >>>>>register that describes the location of the relevant data, or
> > >>>>>just a single "port ID" bit that can be applied to anything?
> > >>>>>
> > >>>>>(Perhaps this isn't so obvious looking at the TRM since it
> > >>>>>doesn't document all registers, and I'm also looking at the
> > >>>>>Tegra30 documentation too, which might be more exposed to this -
> > >>>>>I haven't correlated all the documentation sources to be sure
> > >>>>>though)
> > >>>>
> > >>>>I agree that maybe adding properties for each bit position or
> > >>>>register offset may not work out too well. But I think it still
> > >>>>makes sense to use the base address of the port's registers (see
> > >>>>below). We could of course add some code to determine the index
> > >>>>from the base address at initialization time and reuse the index
> > >>>>where appropriate.
> > >>>
> > >>>To me, working back from address to ID then using the ID to calculate
> > >>>some other addresses seems far more icky than just calculating all the
> > >>>addresses based off of one ID. But, I suppose this doesn't make a huge
> > >>>practical difference.
> > >>
> > >>This really depends on the device vs. no device decision below. If we can
> > >>make it work without needing an extra device for it, then using the index
> > >>is certainly better. However, if we instantiate devices from the DT, then
> > >>we have the address anyway and adding the index as a property would be
> > >>redundant and error prone (what happens if somebody sets the index of the
> > >>port at address 0x80000000 to 2?).
> > >
> > >An additional problem with this is that we'd have to add the following
> > >to the pcie-controller node:
> > >
> > > #address-cells = <1>;
> > > #size-cells = <0>;
> > >
> > >This will conflict with the "ranges" property, because suddenly we can
> > >no longer map the regions properly. Maybe Mitch can comment on whether
> > >this is possible or not?
> > >
> > >To make it clearer what I'm talking about, here's the DT snippet again
> > >(with the compatible property removed from the pci@ nodes because they
> > >are no longer probed by a driver, the "simple-bus" removed from the
> > >pcie-controller node's compatible property removed and its #address-
> > >and #size-cells properties adjusted as described above).
> > >
> > > pcie-controller {
> > > compatible = "nvidia,tegra20-pcie";
> > > reg = <0x80003000 0x00000800 /* PADS registers */
> > > 0x80003800 0x00000200 /* AFI registers */
> > > 0x80004000 0x00100000 /* configuration space */
> > > 0x80104000 0x00100000>; /* extended configuration space */
> > > interrupts = <0 98 0x04 /* controller interrupt */
> > > 0 99 0x04>; /* MSI interrupt */
> > > status = "disabled";
> > >
> > > ranges = <0x80000000 0x80000000 0x00002000 /* 2 root ports */
> > > 0x80400000 0x80400000 0x00010000 /* downstream I/O */
> > > 0x90000000 0x90000000 0x10000000 /* non-prefetchable memory */
> > > 0xa0000000 0xa0000000 0x10000000>; /* prefetchable memory */
> > >
> > > #address-cells = <1>;
> > > #size-cells = <0>;
> > >
> > > pci@0 {
> > > reg = <2>;
> > > status = "disabled";
> > >
> > > #address-cells = <3>;
> > > #size-cells = <2>;
> > >
> > > ranges = <0x81000000 0 0 0x80400000 0 0x00008000 /* I/O */
> > > 0x82000000 0 0 0x90000000 0 0x08000000 /* non-prefetchable memory */
> > > 0xc2000000 0 0 0xa0000000 0 0x08000000>; /* prefetchable memory */
> > >
> > > nvidia,ctrl-offset = <0x110>;
> > > nvidia,num-lanes = <2>;
> > > };
> > >
> > > pci@1 {
> > > reg = <1>;
> > > status = "disabled";
> > >
> > > #address-cells = <3>;
> > > #size-cells = <2>;
> > >
> > > ranges = <0x81000000 0 0 0x80408000 0 0x00008000 /* I/O */
> > > 0x82000000 0 0 0x98000000 0 0x08000000 /* non-prefetchable memory */
> > > 0xc2000000 0 0 0xa8000000 0 0x08000000>; /* prefetchable memory */
> > >
> > > nvidia,ctrl-offset = <0x118>;
> > > nvidia,num-lanes = <2>;
> > > };
> > > };
> > >
> > >AIUI none of the ranges properties are valid anymore, because the bus
> > >represented by pcie-controller no longer reflects the truth, namely that
> > >it translates the CPU address space to the PCI address space.
> > >
> >
> > I think you can use a small-integer port number as an address by
> > defining the intermediate address space properly, and using
> > appropriate ranges above and below. Here's a swag at how that would
> > look.
> >
> > I present three versions, using different choices for the
> > intermediate address space encoding. The intermediate address space
> > may seem somewhat artificial, in that it decouples the "linear pass
> > through of ranges" between the lower PCI address space and the upper
> > CPU address space. But in another sense, it accurately reflects
> > that fact that the bus bridge "slices and dices" that linear address
> > space into non-contiguous pieces.
> >
> > Note that I'm also fixing a problem that I neglected to mention
> > earlier - namely the fact that config space is part of the child PCI
> > address space so it must be passed through.
> >
> > Version A - 3 address cells: In this version, the intermediate
> > address space has 3 cells: port#, address type, offset. Address
> > type is
> > 0 : root port
> > 1 : config space
> > 2 : extended config space
> > 3 : I/O
> > 4 : non-prefetchable memory
> > 5 : prefetchable memory.
> >
> > The third cell "offset" is necessary so that the size field has a
> > number space that can include it.
> >
> > pcie-controller {
> > compatible = "nvidia,tegra20-pcie";
> > reg = <0x80003000 0x00000800 /* PADS registers */
> > 0x80003800 0x00000200>; /* extended configuration space */
> > interrupts = <0 98 0x04 /* controller interrupt */
> > 0 99 0x04>; /* MSI interrupt */
> > status = "disabled";
> >
> > ranges = <0 0 0 0x80000000 0x00001000 /* Root port 0 */
> > 0 1 0 0x80004000 0x00080000 /* Port 0 config space */
> > 0 2 0 0x80104000 0x00080000 /* Port 0 ext config space *
> > 0 3 0 0x80400000 0x00008000 /* Port 0 downstream I/O */
> > 0 4 0 0x90000000 0x08000000 /* Port 0 non-prefetchable memory */
> > 0 5 0 0xa0000000 0x08000000 /* Port 0 prefetchable memory */
> >
> > 1 0 0 0x80001000 0x00001000 /* Root port 1 */
> > 1 1 0 0x80004000 0x00080000 /* Port 1 config space */
> > 1 2 0 0x80184000 0x00080000 /* Port 1 ext config space */
> > 1 3 0 0x80408000 0x00010000 /* Port 1 downstream I/O */
> > 1 4 0 0x98000000 0x08000000 /* Port 1 non-prefetchable memory */
> > 1 5 0 0xa0000000 0x08000000>; /* Port 1 prefetchable memory */
> >
> > #address-cells = <3>;
> > #size-cells = <1>;
> >
> > pci@0 {
> > reg = <0 0 0 0x1000>;
> [...]
> > };
> >
> > pci@1 {
> > reg = <1 0 0 0x1000>;
> [...]
> > };
>
> It seems like this isn't working properly. For some reason both the reg
> property of pci@0 and pci@1 are translated to the same parent address
> 0x80000000. I'll have to investigate where exactly this goes wrong.
So it turns out that while of_read_number() can actually read any number
of cells, only the two least significant are kept because it returns a
u64. Since of_bus_default_map() uses of_read_number() the addresses
<0 0 0> and <1 0 0> are in fact the same. I'm not sure how best to solve
this. I'm not aware of a 128 bit integer type in the kernel, so I guess
the better alternative would be to fix of_bus_default_map() to cope with
#address-cells > 2 properly.
Thierry
Attachment:
pgpGvfrWKjHIg.pgp
Description: PGP signature
