Hi Will, I'm pulling you in at Marc's suggestion. Do you have an opinion
on what the better solution to this problem is?
The executive summary is that Apple SoCs require nGnRnE memory mappings
for MMIO, except for PCI space which uses nGnRE. ARM64 currently uses
nGnRE everywhere. Solutions discussed in this thread and elsewhere include:
1. Something similar to the current hacky patch (which isn't really
intended as a real solution...); change ioremap to default to nGnRnE
using some kind of platform-level flag in the DT, then figure out how to
get the PCI drivers to bypass that. This requires changing almost every
PCI driver, since most of them use plain ioremap().
2. A per-device DT property that tells of_address_to_resource to set a
flag in the struct resource, which is then used by
devm_ioremap_resource/of_iomap to pick a new ioremap_ variant for nGnRnE
(introduce ioremap_np() for nonposted?) (PCI would work with this
inasmuch as it would not support it, and thus fall back to the current
nGnRE default, which is correct for PCI...). This requires changing
DT-binding drivers that we depend on to not use plain ioremap() (if any
do so), but that is a finite subset (unlike PCI which involves
potentially every driver, because thunderbolt).
3. Encode the mapping type in the address of child devices, either
abusing the high bits of the reg or introducing an extra DT cell there,
introduce a new OF bus type that strips those away via a ranges mapping
and sets flags in the struct resource, similar to what the PCI bus does
with its 3-cell ranges, then do as (2) above and make
devm_ioremap_resource/of_iomap use it:
On 09/02/2021 07.57, Arnd Bergmann wrote:
#define MAP_NONPOSTED 0x80000000
arm-io { /* name for adt, should be changed */
compatible = "apple,m1-internal-bus";
#address-cells = <2>; /* or maybe <3> if we want */
#size-cells = <2>;
ranges =
/* on-chip MMIO */
<(MAP_NONPOSTED | 0x2) 0x0 0x2 0x0 0x1 0x0>,
/* first PCI: 2GB control, 4GB memory space */
<(MAP_NONPOSTED | 0x3) 0x80000000 0x3 0x80000000 0x0 0x80000000>,
<0x4 0x0 0x4 0x0 0x1 0x0>,
[...]
The MAP_NONPOSTED flag then gets interpreted by the .translate() and
.get_flags() callbacks of "struct of_bus" in the kernel, where it is put into
a "struct resource" flag, and interpreted when the resource gets mapped.
The PCI host controller nests inside of the node above, and (in theory)
uses the same trick to distinguish between prefetchable and non-prefetchable
memory, except in practice this is handled in device drivers that already
know whether to call ioremap() or ioremap_wc().
4. Introduce a new top-level DT element in the style of reserved-memory,
that describes address ranges and the mapping type to use. This could be
implemented entirely in arch code, having arm64's ioremap look up the
address in a structure populated from this.
As an additional wrinkle, earlycon is almost certainly going to need a
special path to handle this very early, before OF stuff is available; it
also uses fixmap instead of ioremap, which has its own idea about what
type of mapping to use.
--
Hector Martin (marcan@xxxxxxxxx)
Public Key: https://mrcn.st/pub
