On Thu, May 19, 2016 at 12:49 AM, Jiang, Dave <dave.jiang@xxxxxxxxx> wrote: > On Wed, 2016-05-18 at 13:27 +0000, Gavin Guo wrote: >> On Tue, May 17, 2016 at 6:06 PM, Vinod Koul <vinod.koul@xxxxxxxxx> >> wrote: >> > >> > On Mon, May 16, 2016 at 06:08:20PM +0800, Gavin Guo wrote: >> > > >> > > The following error messages can be observed on the Intel >> > > Haswell-E >> > > chipset with v3.13 kernel. After the analysis, I found there is >> > > no >> > > difference in the logic of these error messages in the current >> > > upstream kernel. I also searched the git log and can't find any >> > > commit >> > > which is fix to the error(correct me if I am wrong). The >> > > following is >> > > the detail, and I'll really appreciate if there is any comment. >> > > :) >> > 3.13 is ancient, can you check this on latest kernel >> Thank you for the comment. It's running on the production system. >> However, >> I'll try to figure out if it's possible to test the latest kernel. > > I wonder if you don't have the extended PCI config space access enabled > in your kernel config. Really thanks for your advice. :) I searched the internet about the extended PCI config space and found the link: [Patch v2] Make PCI extended config space (MMCONFIG) a driver opt-in http://lwn.net/Articles/263288/ And I checked the config and found the CONFIG_PCI_MMCONFIG=y. The following string also can be observed in the dmesg: [ 1.419853] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem0x80000000-0x8fffffff] (base 0x80000000) [ 1.419855] PCI: MMCONFIG at [mem 0x80000000-0x8fffffff] reserved in E820 It seems the extended PCI config space is enabled. If there is anything missed? > >> >> > >> > >> > > >> > > >> > > ioatdma 0000:00:04.0: channel error register unreachable >> > > ioatdma 0000:00:04.0: channel enumeration error >> > > ioatdma 0000:00:04.0: Intel(R) I/OAT DMA Engine init failed >> > > ioatdma 0000:00:04.1: channel error register unreachable >> > > ioatdma 0000:00:04.1: channel enumeration error >> > > ioatdma 0000:00:04.1: Intel(R) I/OAT DMA Engine init failed >> > > ... >> > > ioatdma 0000:00:04.7: channel error register unreachable >> > > ioatdma 0000:00:04.7: channel enumeration error >> > > ioatdma 0000:00:04.7: Intel(R) I/OAT DMA Engine init failed >> > > mei_me 0000:00:16.0: initialization failed. >> > > >> > > There are 8 I/OAT DMA controllers on the Haswell-E chipset: >> > > 8086:2f20 ~ 8086:2f27 >> > > 80:04.0 System peripheral: Intel Corporation Haswell-E DMA >> > > Channel 0 (rev 02) >> > > 80:04.1 System peripheral: Intel Corporation Haswell-E DMA >> > > Channel 1 (rev 02) >> > > 80:04.2 System peripheral: Intel Corporation Haswell-E DMA >> > > Channel 2 (rev 02) >> > > 80:04.3 System peripheral: Intel Corporation Haswell-E DMA >> > > Channel 3 (rev 02) >> > > 80:04.4 System peripheral: Intel Corporation Haswell-E DMA >> > > Channel 4 (rev 02) >> > > 80:04.5 System peripheral: Intel Corporation Haswell-E DMA >> > > Channel 5 (rev 02) >> > > 80:04.6 System peripheral: Intel Corporation Haswell-E DMA >> > > Channel 6 (rev 02) >> > > 80:04.7 System peripheral: Intel Corporation Haswell-E DMA >> > > Channel 7 (rev 02) >> > > >> > > Analysis: >> > > The bug happens when the driver is resetting DMA controller, this >> > > is >> > > the sequence: The function, ioat_pci_probe, is called when the >> > > DMA >> > > controller is detected by the PCI bus. Then, >> > > ioat3_dma_probe -> ioat_probe -> ioat2_enumerate_channels -> >> > > ioat3_reset_hw. The following code can be found in the >> > > ioat3_reset_hw: >> > > >> > > drivers/dma/ioat/dma_v3.c: >> > > chanerr = readl(chan->reg_base + IOAT_CHANERR_OFFSET); >> > > writel(chanerr, chan->reg_base + IOAT_CHANERR_OFFSET); >> > > ... >> > > err = pci_read_config_dword(pdev, >> > > IOAT_PCI_CHANERR_INT_OFFSET, &chanerr); >> > > if (err) { >> > > dev_err(&pdev->dev, >> > > "channel error register unreachable\n"); >> > > return err; >> > > } >> > > >> > > Obviously, there are something wrong in the channel error >> > > register >> > > reset process. Then all the way back to ioat_probe(). Because the >> > > error happens, the dma->chancnt will be set to 0: >> > > >> > > drivers/dma/ioat/dma.c: >> > > if (!dma->chancnt) { >> > > dev_err(dev, "channel enumeration error\n"); >> > > goto err_setup_interrupts; >> > > } >> > > >> > > Finally back to ioat_pci_probe: >> > > >> > > drivers/dma/ioat/pci.c: >> > > err = ioat3_dma_probe(device, ioat_dca_enabled); >> > > else >> > > return -ENODEV; >> > > >> > > if (err) { >> > > dev_err(dev, "Intel(R) I/OAT DMA Engine init >> > > failed\n"); >> > > return -ENODEV; >> > -- >> > ~Vinod
Attachment:
ioatdma_dmesg
Description: Binary data
