This is a draft patchset trying to fix the issue that AMD IOMMU doesn't
work well in kdump kernel. The patch arrangement is not foraml, just
presenting what I have done and what's the problem being encountered
currently.
It contains 3 parts.
1) Clean up patch
Patch 1/10, 2/10, 3/10 are code clean up patches, which later part will
be based on.
2) IO page mapping
Patch 4/10 ~ 9/10
.> Checking if it's in kdump kernel and previously enabled
.> If yes do below operatons:
.> Do not disable amd iommu and do not touch dev tables before coping old dev tables
.> Copy dev table form old kernel and set the old domain id in amd_iommu_pd_alloc_bitmap
.> Don't call update_domain() to set domain->pt_root to dev entries before device driver initialization.
.> Reset the pre-enabled status when the first __map_single() is called during device driver init
3)interrupt remapping
Patch 10/10
.> I didn't think of this well. Now I only copy the old irq table when it first calls get_irq_table().
This need people's suggestion. Maybe not correct old irq table copy cause kdump kernel hang.
Now there are several problems I got:
Now there's always a hang when go into kdump kernel so that I can't test
futher if command buffer/envent buffer need be copied and where flush need
be called.
Kdump kernel hang and dump the call trace to show it happened in check_timer.
This is similar as people found when they debugged intel iommu issue.
http://lists.infradead.org/pipermail/kexec/2014-December/013137.html
[ 12.296525] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[ 12.302513] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.2.0+ #18
[ 12.308500] Hardware name: AMD Dinar/Dinar, BIOS RDN1505B 06/05/2013
[ 12.314832] 0000000000000000 0000000085c693e9 ffff880030d6fd58
ffffffff8139746f
[ 12.322239] 00000000000000a0 ffff880030d6fd90 ffffffff814b4813
ffff880030d283c0
[ 12.329645] ffff880030d2e100 0000000000000002 0000000000000000
ffff880030c29808
[ 12.337052] Call Trace:
[ 12.339493] [<ffffffff8139746f>] dump_stack+0x44/0x55
[ 12.344616] [<ffffffff814b4813>] modify_irte+0x23/0xc0
[ 12.349827] [<ffffffff814b48cc>] irq_remapping_deactivate+0x1c/0x20
[ 12.356162] [<ffffffff814b48de>] irq_remapping_activate+0xe/0x10
[ 12.362238] [<ffffffff810fa6b1>] irq_domain_activate_irq+0x41/0x50
[ 12.368486] [<ffffffff810fa69b>] irq_domain_activate_irq+0x2b/0x50
[ 12.374736] [<ffffffff81d6ccbb>] setup_IO_APIC+0x33e/0x7e4
[ 12.380294] [<ffffffff81052039>] ? clear_IO_APIC+0x39/0x60
[ 12.385853] [<ffffffff81d6b82c>] apic_bsp_setup+0xa1/0xac
[ 12.391323] [<ffffffff81d69463>] native_smp_prepare_cpus+0x25f/0x2db
[ 12.397747] [<ffffffff81d550ee>] kernel_init_freeable+0xc9/0x228
[ 12.403824] [<ffffffff81762370>] ? rest_init+0x80/0x80
[ 12.409034] [<ffffffff8176237e>] kernel_init+0xe/0xe0
[ 12.414158] [<ffffffff8176e19f>] ret_from_fork+0x3f/0x70
[ 12.419541] [<ffffffff81762370>] ? rest_init+0x80/0x80
[ 12.424751] modify_irte devid: 00:14.0 index: 2, vector:48
[ 12.440491] Kernel panic - not syncing: timer doesn't work through
Interrupt-remapped IO-APIC
[ 12.449022] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.2.0+ #18
[ 12.455008] Hardware name: AMD Dinar/Dinar, BIOS RDN1505B 06/05/2013
[ 12.461340] 0000000000000000 0000000085c693e9 ffff880030d6fd58
ffffffff8139746f
[ 12.468753] ffffffff81a3cdf8 ffff880030d6fde0 ffffffff8119e921
0000000000000008
[ 12.476165] ffff880030d6fdf0 ffff880030d6fd88 0000000085c693e9
ffffffff813a41b5
[ 12.483577] Call Trace:
[ 12.486018] [<ffffffff8139746f>] dump_stack+0x44/0x55
[ 12.491142] [<ffffffff8119e921>] panic+0xd3/0x20b
[ 12.495919] [<ffffffff813a41b5>] ? delay_tsc+0x25/0x60
[ 12.501129] [<ffffffff814bfaba>] panic_if_irq_remap+0x1a/0x20
[ 12.506947] [<ffffffff81d6ccf2>] setup_IO_APIC+0x375/0x7e4
[ 12.512503] [<ffffffff81052039>] ? clear_IO_APIC+0x39/0x60
[ 12.518060] [<ffffffff81d6b82c>] apic_bsp_setup+0xa1/0xac
[ 12.523530] [<ffffffff81d69463>] native_smp_prepare_cpus+0x25f/0x2db
[ 12.529952] [<ffffffff81d550ee>] kernel_init_freeable+0xc9/0x228
[ 12.536030] [<ffffffff81762370>] ? rest_init+0x80/0x80
[ 12.541238] [<ffffffff8176237e>] kernel_init+0xe/0xe0
[ 12.546361] [<ffffffff8176e19f>] ret_from_fork+0x3f/0x70
[ 12.551745] [<ffffffff81762370>] ? rest_init+0x80/0x80
[ 12.556957] Rebooting in 10 seconds..
The problem happened in check_timer(). Seems timer interrupt doesn't
work well after modify_irte(). I don't know why it happened. Though I
have copied the old irte tables.
Baoquan He (10):
iommu/amd: Use standard bitmap operation to set bitmap
iommu/amd: Adjust functons which get first/last devid by reading pci
config
iommu/amd: Get the first/last device of iommu earlier
iommu/amd: Detect pre enabled translation
iommu/amd: Add function copy_dev_tables
iommu/amd: Add functions copy_command_buffer/copy_event_buffer
iommu/amd: copy old dev tables and do not change it
iommu/amd: Do not update the information of domain to devtables before
device driver init
iommu/amd: Clear the iommu pre enabled setting
iommu/amd: Copy the old ir table
drivers/iommu/amd_iommu.c | 31 ++++--
drivers/iommu/amd_iommu_init.c | 205 +++++++++++++++++++++++++++++++---------
drivers/iommu/amd_iommu_proto.h | 4 +
drivers/iommu/amd_iommu_types.h | 3 +
4 files changed, 189 insertions(+), 54 deletions(-)
--
2.4.0
