On Fri, Sep 20, 2024 at 01:58:23AM -0700, Palmer Dabbelt wrote:
> On Tue, 27 Aug 2024 16:10:20 PDT (-0700), dfustini@xxxxxxxxxxxxxxx wrote:
> > On Tue, Aug 27, 2024 at 02:36:11PM +0300, Vladimir Kondratiev wrote:
> > > It is not necessary any RISCV platform has ZONE_DMA32.
> > >
> > > Example - if platform has no DRAM in [0..4G] region,
> > > it will report failure like below each boot.
> > >
> > > [ 0.088709] swapper/0: page allocation failure: order:7, mode:0xcc4(GFP_KERNEL|GFP_DMA32), nodemask=(null),cpuset=/
> > > [ 0.088832] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted 6.11.0-rc5 #30
> > > [ 0.088864] Call Trace:
> > > [ 0.088869] [<ffffffff800059f2>] dump_backtrace+0x1c/0x24
> > > [ 0.088910] [<ffffffff805f328c>] show_stack+0x2c/0x38
> > > [ 0.088957] [<ffffffff805fd800>] dump_stack_lvl+0x52/0x74
> > > [ 0.088987] [<ffffffff805fd836>] dump_stack+0x14/0x1c
> > > [ 0.089010] [<ffffffff801a23a8>] warn_alloc+0xf4/0x176
> > > [ 0.089041] [<ffffffff801a3052>] __alloc_pages_noprof+0xc28/0xcb4
> > > [ 0.089067] [<ffffffff80086eda>] atomic_pool_expand+0x62/0x1f8
> > > [ 0.089090] [<ffffffff8080d674>] __dma_atomic_pool_init+0x46/0x9e
> > > [ 0.089115] [<ffffffff8080d762>] dma_atomic_pool_init+0x96/0x11c
> > > [ 0.089139] [<ffffffff80002146>] do_one_initcall+0x5c/0x1b2
> > > [ 0.089158] [<ffffffff8080127c>] kernel_init_freeable+0x214/0x274
> > > [ 0.089190] [<ffffffff805fefd8>] kernel_init+0x1e/0x10a
> > > [ 0.089209] [<ffffffff8060748a>] ret_from_fork+0xe/0x1c
> > >
> > > Signed-off-by: Vladimir Kondratiev <vladimir.kondratiev@xxxxxxxxxxxx>
> > > ---
> > > arch/riscv/Kconfig | 2 +-
> > > mm/Kconfig | 2 +-
> > > 2 files changed, 2 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > index 0f3cd7c3a436..94a573112625 100644
> > > --- a/arch/riscv/Kconfig
> > > +++ b/arch/riscv/Kconfig
> > > @@ -50,6 +50,7 @@ config RISCV
> > > select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST
> > > select ARCH_HAS_UBSAN
> > > select ARCH_HAS_VDSO_DATA
> > > + select ARCH_HAS_ZONE_DMA_SET if 64BIT
> > > select ARCH_KEEP_MEMBLOCK if ACPI
> > > select ARCH_MHP_MEMMAP_ON_MEMORY_ENABLE if 64BIT && MMU
> > > select ARCH_OPTIONAL_KERNEL_RWX if ARCH_HAS_STRICT_KERNEL_RWX
> > > @@ -200,7 +201,6 @@ config RISCV
> > > select THREAD_INFO_IN_TASK
> > > select TRACE_IRQFLAGS_SUPPORT
> > > select UACCESS_MEMCPY if !MMU
> > > - select ZONE_DMA32 if 64BIT
> > >
> > > config CLANG_SUPPORTS_DYNAMIC_FTRACE
> > > def_bool CC_IS_CLANG
> > > diff --git a/mm/Kconfig b/mm/Kconfig
> > > index b72e7d040f78..97c85da98e89 100644
> > > --- a/mm/Kconfig
> > > +++ b/mm/Kconfig
> > > @@ -1032,7 +1032,7 @@ config ZONE_DMA
> > > config ZONE_DMA32
> > > bool "Support DMA32 zone" if ARCH_HAS_ZONE_DMA_SET
> > > depends on !X86_32
> > > - default y if ARM64
> > > + default y if ARM64 || (RISCV && 64BIT)
> > >
> > > config ZONE_DEVICE
> > > bool "Device memory (pmem, HMM, etc...) hotplug support"
> > > --
> > > 2.37.3
> > >
> >
> > Reviewed-by: Drew Fustini <dfustini@xxxxxxxxxxxxxxx>
> >
> > Thanks for sending this patch as I've also encountered that annoying
> > error on systems with DRAM above 4GB.
> >
> > I tested this patch by changing the qemu virt machine to have DRAM
> > starting at 2^32:
> >
> > diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
> > index cef41c150aaf..3033a2560edb 100644
> > --- a/hw/riscv/virt.c
> > +++ b/hw/riscv/virt.c
> > @@ -87,7 +87,7 @@ static const MemMapEntry virt_memmap[] = {
> > [VIRT_IMSIC_S] = { 0x28000000, VIRT_IMSIC_MAX_SIZE },
> > [VIRT_PCIE_ECAM] = { 0x30000000, 0x10000000 },
> > [VIRT_PCIE_MMIO] = { 0x40000000, 0x40000000 },
> > - [VIRT_DRAM] = { 0x80000000, 0x0 },
> > + [VIRT_DRAM] = { 0x100000000, 0x0 },
> > };
> >
> > /* PCIe high mmio is fixed for RV32 */
>
> IIRC the ZONE_DMA32 stuff existed for some of the early SiFive systems,
> where the expansion daughterboard's PCIe controller (via a Xilinx FPGA)
> could only handle 32-bit DMA addreses. I think there's a similar quirk in
> the Microsemi PCIe controller on the PolarFire boards, but Conor would know
> for sure.
I don't think this patch would affect those systems that need ZONE_DMA2.
I believe it just makes it possible to disable it in the kernel config.
The platform I'm working on has no memory below 4GB and all the PCIe
devices that I care about are not 32-bit constrained. Therefore I just
want to be able to turn it off in my .config.
Thanks,
Drew
