Hi Lukas,
>The solution is to acquire a ref on each device in add_error_device():
I will give try and update if it works .
regards,
Gokul
On Thu, Aug 2, 2018 at 2:16 PM, Lukas Wunner <lukas@xxxxxxxxx> wrote:
On Thu, Aug 02, 2018 at 12:59:18PM +0530, gokul cg wrote:
> I am suspecting a possible race condition in the kernel between PCI driver
> and AER handling.
>
> Because of the same kernel panic happens from worker thread which handles
> bottom half of aer irq.
>
> I am seeing this issue when I suddenly power off PCI card which
> supports/enabled PCIE AER error reporting.
>
> While powering off PCI device, AER driver will get AER IRQ for the device,
> from AER IRQ handler, it will cache AER error code and schedule worker
> thread to handle error.
>
> The PCIe device will get removed from PCI tree before worker thread
> completes its task and kernel panic is happening when worker thread tries
> to access PCI device's config space.
>
> #5 [ffff88027469fc70] general_protection at ffffffff8176cdf2
> [exception RIP: pci_bus_read_config_dword+100]
> #6 [ffff88027469fd50] pci_find_next_ext_capability at ffffffff81345d7b
> #7 [ffff88027469fd90] pci_find_ext_capability at ffffffff81347225
> #8 [ffff88027469fda0] get_device_error_info at ffffffff81356c4d
> #9 [ffff88027469fdd0] aer_isr at ffffffff81357a38
> #10 [ffff88027469fe28] process_one_work at ffffffff8105d4c0
> #11 [ffff88027469fe70] worker_thread at ffffffff8105e251
> #12 [ffff88027469fed0] kthread at ffffffff81064260
> #13 [ffff88027469ff50] ret_from_fork at ffffffff81773a38
>
> I have tested it on kernel 3.10 . But from source i could see that this
> case is still relevant for latest Linux source .
I'm not really familiar with the AER driver, but the problem is
actually easy to spot:
find_source_device() walks the hierarchy and saves a pointer to
pci_dev's in an array. That array is later traversed and the
pci_dev's are accessed.
The solution is to acquire a ref on each device in add_error_device():
- e_info->dev[e_info->error_dev_num] = dev;
+ e_info->dev[e_info->error_dev_num] = pci_dev_get(dev);
Then release the ref aer_process_err_devices() by calling pci_dev_put().
I believe there's an ongoing refactoring of the AER driver and the
issue may be addressed in the course of it, but as a quick fix for
an ancient v3.10 kernel, the above should do the trick.
HTH,
Lukas
