On Wed, Oct 05, 2016 at 08:37:01AM -0700, Christoph Hellwig wrote:
> On Thu, Sep 29, 2016 at 04:28:02PM -0600, Jason Gunthorpe wrote:
> > In the kernel the swapping behavior of 'writel' is architecture (and
> > sometimes even platform) specific. Looking at the kernel headers
> > suggests that BE PPC needs an explicit swap and BE ARM does not (eg
> > the hardware swaps). Maybe. There are a lot of layers of macros in
> > this area..
>
> read{s,l,q} and write{s,l,q} in the kernel always perform an implicit
> bytestap, that is the device is expected to use a LE layout. This
> is not architecture specific, otherwise hell would break lose.
Right, stated differently, the kernel requires that writel()/etc
always produce the same PCI-E packet on the wire. (eg writel(1)
produces a TLP with bit 0 of the data payload set)
But each arch implements its own set of instructions to get there,
that is what I mean by architecture specific.
> But none of this has an affect on userspace mappings of registers, as
> those don't go through the kernel read{s,l,q} and write{s,l,q}
> helpers,
Not going through the kernel's writel is the whole problem. The writel
helper generates the arch-specific instruction sequence required to
issue generate the required PCI-E packet.
Today (at best, some drivers do not even do this) our userspace
assumes all archs implement writel as:
*(u32 *)reg = cpu_to_le32(val);
Which is a good start, but not portable to every arch the kernel
supports.
> Btw, one thing that would be useful is to introduce endiane annoatations
> for sparse in rdma-core. I've added these to a few userspace projects
> already, here is the most recent example:
>
> https://github.com/linux-nvme/nvme-cli/commit/126b8beb35b76fe9f61b137e34aea66b64c07b25
Yeah, it would be nice to get that working too. I guess we need to
standardize on the cpu_to_xx macro style as a first step?
The __iomem annotation would be nice as well.
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
