Hello Lennert,
On Tue, Jan 16, 2024 at 02:27:40PM +0200, Lennert Buytenhek wrote:
> Hi,
>
> On kernel 6.6.x, with an ASMedia ASM1062 (AHCI) controller, on an
> ASUSTeK Pro WS WRX80E-SAGE SE WIFI mainboard, PCI ID 1b21:0612 and
> subsystem ID 1043:858d, I got a total apparent controller hang,
> rendering the two attached SATA devices unavailable, that was
> immediately preceded by the following kernel messages:
>
> [Thu Jan 4 23:12:54 2024] ahci 0000:28:00.0: Using 64-bit DMA addresses
> [Thu Jan 4 23:12:54 2024] ahci 0000:28:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0035 address=0x7fffff00000 flags=0x0000]
> [Thu Jan 4 23:12:54 2024] ahci 0000:28:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0035 address=0x7fffff00300 flags=0x0000]
> [Thu Jan 4 23:12:54 2024] ahci 0000:28:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0035 address=0x7fffff00380 flags=0x0000]
> [Thu Jan 4 23:12:54 2024] ahci 0000:28:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0035 address=0x7fffff00400 flags=0x0000]
> [Thu Jan 4 23:12:54 2024] ahci 0000:28:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0035 address=0x7fffff00680 flags=0x0000]
> [Thu Jan 4 23:12:54 2024] ahci 0000:28:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0035 address=0x7fffff00700 flags=0x0000]
>
> It seems as if the controller has problems with 64-bit DMA addresses,
> and the comments around the source of the message in
> drivers/iommu/dma-iommu.c seem to point into that same direction:
>
> /*
> * Try to use all the 32-bit PCI addresses first. The original SAC vs.
> * DAC reasoning loses relevance with PCIe, but enough hardware and
> * firmware bugs are still lurking out there that it's safest not to
> * venture into the 64-bit space until necessary.
> *
> * If your device goes wrong after seeing the notice then likely either
> * its driver is not setting DMA masks accurately, the hardware has
> * some inherent bug in handling >32-bit addresses, or not all the
> * expected address bits are wired up between the device and the IOMMU.
> */
> if (dma_limit > DMA_BIT_MASK(32) && dev->iommu->pci_32bit_workaround) {
> iova = alloc_iova_fast(iovad, iova_len,
> DMA_BIT_MASK(32) >> shift, false);
> if (iova)
> goto done;
>
> dev->iommu->pci_32bit_workaround = false;
> dev_notice(dev, "Using %d-bit DMA addresses\n", bits_per(dma_limit));
> }
The DMA mask is set here:
https://github.com/torvalds/linux/blob/v6.7/drivers/ata/ahci.c#L967
And should be called using:
hpriv->cap & HOST_CAP_64
https://github.com/torvalds/linux/blob/v6.7/drivers/ata/ahci.c#L1929
Where hpriv->cap is capabilities reported by the AHCI controller itself.
So it definitely seems like your controller supports 64-bit addressing.
I guess it could be some problem with your BIOS.
Have you tried updating your BIOS?
If that does not work, perhaps you could try this (completely untested) patch:
(You might need to modify the strings to match the exact strings reported by
your BIOS.)
If it works, we need to add a specific BIOS version too, see e.g.
https://github.com/torvalds/linux/blob/v6.7/drivers/ata/ahci.c#L1310
diff --git a/drivers/ata/ahci.c b/drivers/ata/ahci.c
index 3a5f3255f51b..35dead43142c 100644
--- a/drivers/ata/ahci.c
+++ b/drivers/ata/ahci.c
@@ -1034,6 +1034,30 @@ static void ahci_p5wdh_workaround(struct ata_host *host)
}
}
+static bool ahci_broken_64_bit(struct pci_dev *pdev)
+{
+ static const struct dmi_system_id sysids[] = {
+ {
+ .ident = "ASUS Pro WS WRX80E-SAGE",
+ .matches = {
+ DMI_MATCH(DMI_BOARD_VENDOR,
+ "ASUSTeK Computer INC."),
+ DMI_MATCH(DMI_BOARD_NAME, "Pro WS WRX80E-SAGE"),
+ },
+ },
+ { }
+ };
+ const struct dmi_system_id *dmi = dmi_first_match(sysids);
+
+ if (!dmi)
+ return false;
+
+ dev_warn(&pdev->dev, "%s: forcing 32bit DMA, update BIOS\n",
+ dmi->ident);
+
+ return true;
+}
+
/*
* Macbook7,1 firmware forcibly disables MCP89 AHCI and changes PCI ID when
* booting in BIOS compatibility mode. We restore the registers but not ID.
@@ -1799,6 +1823,10 @@ static int ahci_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
if (ahci_broken_devslp(pdev))
hpriv->flags |= AHCI_HFLAG_NO_DEVSLP;
+ /* must set flag prior to save config in order to take effect */
+ if (ahci_broken_64_bit(pdev))
+ hpriv->flags |= AHCI_HFLAG_32BIT_ONLY;
+
#ifdef CONFIG_ARM64
if (pdev->vendor == PCI_VENDOR_ID_HUAWEI &&
pdev->device == 0xa235 &&
