On 09/14/2020 05:46 PM, Huacai Chen wrote:
Hi, Tiezhu,
On Mon, Sep 14, 2020 at 5:30 PM Tiezhu Yang <yangtiezhu@xxxxxxxxxxx> wrote:
On 09/14/2020 04:52 PM, Huacai Chen wrote:
Hi, Tiezhu,
How do you test kexec? kexec -e or systemctl kexec? Or both?
kexec -l vmlinux --append="root=/dev/sda2 console=ttyS0,115200"
kexec -e
So you haven't tested "systemctl kexec"?
Yes, the distro I used is Loongnix which has not kexec service now.
Is there any problem when use systemctl kexec? If you have more details,
please let me know.
Huacai
P.S., Please also CC my gmail (chenhuacai@xxxxxxxxx) since lemote.com
has some communication problems.
OK, no problem.
Huacai
陈华才江苏航天龙梦信息技术有限公司/研发中心/软件部 ------------------ Original ------------------From: "Tiezhu Yang"<yangtiezhu@xxxxxxxxxxx>;Date: Mon, Sep 14, 2020 03:57 PMTo: "Bjorn Helgaas"<bhelgaas@xxxxxxxxxx>; Cc: "linux-pci"<linux-pci@xxxxxxxxxxxxxxx>; "linux-kernel"<linux-kernel@xxxxxxxxxxxxxxx>; "Rafael J. Wysocki"<rafael.j.wysocki@xxxxxxxxx>; "Konstantin Khlebnikov"<khlebnikov@xxxxxxxxxx>; "Khalid Aziz"<khalid.aziz@xxxxxxxxxx>; "Vivek Goyal"<vgoyal@xxxxxxxxxx>; "Lukas Wunner"<lukas@xxxxxxxxx>; "Oliver O'Halloran"<oohall@xxxxxxxxx>; "Huacai Chen"<chenhc@xxxxxxxxxx>; "Jiaxun Yang"<jiaxun.yang@xxxxxxxxxxx>; "Xuefeng Li"<lixuefeng@xxxxxxxxxxx>; Subject: [RFC PATCH v3] PCI/portdrv: Only disable Bus Master on kexec reboot and connected PCI devices After commit 745be2e700cd ("PCIe: portdrv: call pci_disable_device
during remove") and commit cc27b735ad3a ("PCI/portdrv: Turn off PCIe
services during shutdown"), it also calls pci_disable_device() during
shutdown, this leads to shutdown or reboot failure occasionally due to
clear PCI_COMMAND_MASTER on the device in do_pci_disable_device().
drivers/pci/pci.c
static void do_pci_disable_device(struct pci_dev *dev)
{
u16 pci_command;
pci_read_config_word(dev, PCI_COMMAND, &pci_command);
if (pci_command & PCI_COMMAND_MASTER) {
pci_command &= ~PCI_COMMAND_MASTER;
pci_write_config_word(dev, PCI_COMMAND, pci_command);
}
pcibios_disable_device(dev);
}
When remove "pci_command &= ~PCI_COMMAND_MASTER;", it can work well when
shutdown or reboot.
As Oliver O'Halloran said, no need to call pci_disable_device() when
actually shutting down, but we should call pci_disable_device() before
handing over to the new kernel on kexec reboot, so we can do some
condition checks which are already executed afterwards by the function
pci_device_shutdown(), this is done by commit 4fc9bbf98fd6 ("PCI: Disable
Bus Master only on kexec reboot") and commit 6e0eda3c3898 ("PCI: Don't try
to disable Bus Master on disconnected PCI devices").
drivers/pci/pci-driver.c
static void pci_device_shutdown(struct device *dev)
{
...
if (drv && drv->shutdown)
drv->shutdown(pci_dev);
/*
* If this is a kexec reboot, turn off Bus Master bit on the
* device to tell it to not continue to do DMA. Don't touch
* devices in D3cold or unknown states.
* If it is not a kexec reboot, firmware will hit the PCI
* devices with big hammer and stop their DMA any way.
*/
if (kexec_in_progress && (pci_dev->current_state <= PCI_D3hot))
pci_clear_master(pci_dev);
}
[ 36.159446] Call Trace:
[ 36.241688] [<ffffffff80211434>] show_stack+0x9c/0x130
[ 36.326619] [<ffffffff80661b70>] dump_stack+0xb0/0xf0
[ 36.410403] [<ffffffff806a8240>] pcie_portdrv_shutdown+0x18/0x78
[ 36.495302] [<ffffffff8069c6b4>] pci_device_shutdown+0x44/0x90
[ 36.580027] [<ffffffff807aac90>] device_shutdown+0x130/0x290
[ 36.664486] [<ffffffff80265448>] kernel_power_off+0x38/0x80
[ 36.748272] [<ffffffff80265634>] __do_sys_reboot+0x1a4/0x258
[ 36.831985] [<ffffffff80218b90>] syscall_common+0x34/0x58
Signed-off-by: Tiezhu Yang <yangtiezhu@xxxxxxxxxxx>
---
drivers/pci/pcie/portdrv_core.c | 1 -
drivers/pci/pcie/portdrv_pci.c | 14 +++++++++++++-
2 files changed, 13 insertions(+), 2 deletions(-)
diff --git a/drivers/pci/pcie/portdrv_core.c b/drivers/pci/pcie/portdrv_core.c
index 50a9522..1991aca 100644
--- a/drivers/pci/pcie/portdrv_core.c
+++ b/drivers/pci/pcie/portdrv_core.c
@@ -491,7 +491,6 @@ void pcie_port_device_remove(struct pci_dev *dev)
{
device_for_each_child(&dev->dev, NULL, remove_iter);
pci_free_irq_vectors(dev);
- pci_disable_device(dev);
}
/**
diff --git a/drivers/pci/pcie/portdrv_pci.c b/drivers/pci/pcie/portdrv_pci.c
index 3a3ce40..cab37a8 100644
--- a/drivers/pci/pcie/portdrv_pci.c
+++ b/drivers/pci/pcie/portdrv_pci.c
@@ -143,6 +143,18 @@ static void pcie_portdrv_remove(struct pci_dev *dev)
}
pcie_port_device_remove(dev);
+ pci_disable_device(dev);
+}
+
+static void pcie_portdrv_shutdown(struct pci_dev *dev)
+{
+ if (pci_bridge_d3_possible(dev)) {
+ pm_runtime_forbid(&dev->dev);
+ pm_runtime_get_noresume(&dev->dev);
+ pm_runtime_dont_use_autosuspend(&dev->dev);
+ }
+
+ pcie_port_device_remove(dev);
}
static pci_ers_result_t pcie_portdrv_error_detected(struct pci_dev *dev,
@@ -211,7 +223,7 @@ static struct pci_driver pcie_portdriver = {
.probe = pcie_portdrv_probe,
.remove = pcie_portdrv_remove,
- .shutdown = pcie_portdrv_remove,
+ .shutdown = pcie_portdrv_shutdown,
.err_handler = &pcie_portdrv_err_handler,
--
2.1.0
