On Thu, Feb 16, 2017 at 04:35:31PM +0100, Paolo Abeni wrote:
> The skbs transmitted via ipoib_send() are freed only if there are
> 16 or more outstanding work requests or if the send queue is full.
>
> If there is very little networking activity, the transmitted skbs
> can be held by the device driver for an unlimited amount of time,
> starving other subsystems.
>
> E.g. assuming the ipv6 is enabled, with the following sequence:
>
> systemctl start firewalld
> modprobe ib_ipoib
> ip addr add dev ib0 fc00::1/64
> systemctl stop firewalld
>
> a cpu will hang: rmmod conntrack will keep a core busy
> spinning for nf_conntrack_untracked going to 0, since some ICMP6
> ND packets are generated and transmitted when the ipv6 address
> is attached to the device, and such packets get a notrack ct
> entry.
>
> This change address the issue introducing a periodic timer performing
> "garbage collection" on the send ring at low frequency (once every
> second).
>
> This new timer runs independently from the currently used poll_timer,
> so that no additional delay is introduced to clean the ring after
> errors or ring full event.
>
> Reported-by: Thomas Cameron <tcameron@xxxxxxxxxx>
> Fixes: f56bcd801356 ("IPoIB: Use separate CQ for UD send completions")
> Signed-off-by: Paolo Abeni <pabeni@xxxxxxxxxx>
> ---
> drivers/infiniband/ulp/ipoib/ipoib.h | 2 ++
> drivers/infiniband/ulp/ipoib/ipoib_ib.c | 35 +++++++++++++++++++++++++++------
> 2 files changed, 31 insertions(+), 6 deletions(-)
>
Hi Paolo,
This patch crashes in our verification system on Mellanox CX3 HCA.
[ 105.569758] console [netcon0] enabled
[ 105.569801] netconsole: network logging started
[ 105.570316] sysrq: SysRq : Changing Loglevel
[ 105.570393] sysrq: Loglevel set to 8
[ 127.303248] IPv6: ADDRCONF(NETDEV_UP): ib0: link is not ready
[ 135.501051] BUG: unable to handle kernel NULL pointer dereference at (null)
[ 135.501182] IP: (null)
[ 135.501231] PGD 34d5e067
[ 135.501231] PUD 34eae067
[ 135.501311] PMD 0
[ 135.501351]
[ 135.501431] Oops: 0010 [#1] SMP
[ 135.501491] Modules linked in: netconsole nfsv3 nfs fscache rdma_ucm ib_ucm
rdma_cm iw_cm ib_ipoib ib_cm ib_uverbs ib_umad mlx4_en ptp pps_core ib_core
dm_mirror dm_region_hash dm_log dm_mod ppdev pcspkr nfsd parport_pc parport
i2c_piix4 acpi_cpufreq auth_rpcgss nfs_acl lockd grace sunrpc ip_tables
ata_generic cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops
ttm serio_raw pata_acpi drm virtio_console mlx4_core e1000 devlink i2c_core
floppy ata_piix [last unloaded: mlx4_ib]
[ 135.501812] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.10.0-rc8-2017-02-16_18-01-48_Paolo_Abeni__pabeni_redhat_com #1
[ 135.501895] Hardware name: Red Hat KVM, BIOS Bochs 01/01/2011
[ 135.501985] task: ffff88013a429480 task.stack: ffffc90000694000
[ 135.502057] RIP: 0010: (null)
[ 135.502106] RSP: 0018:ffff88013fcc3e28 EFLAGS: 00010286
[ 135.502155] RAX: ffff880138dd0c00 RBX: ffff880138bac900 RCX: 00000000fffd7780
[ 135.502221] RDX: ffff880138bace88 RSI: 0000000000000010 RDI: ffff880138dd1400
[ 135.502285] RBP: ffff88013fcc3e78 R08: ffff88013fcd0630 R09: ffff88013fcc3ef8
[ 135.502357] R10: 0000000000000200 R11: 0000000000000020 R12: ffff880138bace88
[ 135.502423] R13: 0000000000000001 R14: 0000000000000003 R15: ffff880138bac000
[ 135.502487] FS: 0000000000000000(0000) GS:ffff88013fcc0000(0000) knlGS:0000000000000000
[ 135.502558] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 135.502615] CR2: 0000000000000000 CR3: 000000008d77b000 CR4: 00000000000006e0
[ 135.502675] Call Trace:
[ 135.502711] <IRQ>
[ 135.502766] ? poll_tx+0x39/0x250 [ib_ipoib]
[ 135.502821] ? cpu_load_update+0xdc/0x150
[ 135.502863] ipoib_tx_gc_timer_func+0x115/0x130 [ib_ipoib]
[ 135.502910] ? poll_tx+0x250/0x250 [ib_ipoib]
[ 135.502951] call_timer_fn+0x35/0x140
[ 135.502994] run_timer_softirq+0x1d7/0x460
[ 135.503034] ? kvm_sched_clock_read+0x1e/0x30
[ 135.504133] ? sched_clock+0x9/0x10
[ 135.505240] ? sched_clock_cpu+0x72/0xa0
[ 135.506293] __do_softirq+0xd7/0x2a8
[ 135.507328] irq_exit+0xb5/0xc0
[ 135.508339] smp_apic_timer_interrupt+0x3d/0x50
[ 135.509342] apic_timer_interrupt+0x89/0x90
[ 135.510346] RIP: 0010:native_safe_halt+0x6/0x10
[ 135.511360] RSP: 0018:ffffc90000697ea0 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff10
[ 135.512405] RAX: 0000000000000000 RBX: ffff88013a429480 RCX: 0000000000000000
[ 135.513457] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
[ 135.514496] RBP: ffffc90000697ea0 R08: 00000000d5555b28 R09: 0000000000000000
[ 135.515535] R10: 0000000000000200 R11: ffffc900022b7ef0 R12: 0000000000000003
[ 135.516579] R13: ffff88013a429480 R14: 0000000000000000 R15: 0000000000000000
[ 135.517620] </IRQ>
[ 135.518650] default_idle+0x1e/0xd0
[ 135.519672] arch_cpu_idle+0xf/0x20
[ 135.520673] default_idle_call+0x35/0x40
[ 135.521678] do_idle+0x15b/0x200
[ 135.522687] cpu_startup_entry+0x1d/0x30
[ 135.523706] start_secondary+0x103/0x130
[ 135.524734] start_cpu+0x14/0x14
[ 135.525793] Code: Bad RIP value.
[ 135.526798] RIP: (null) RSP: ffff88013fcc3e28
[ 135.527767] CR2: 0000000000000000
[ 135.528727] ---[ end trace 8acb66738f095ba7 ]---
[ 135.529639] Kernel panic - not syncing: Fatal exception in interrupt
[ 135.531590] Kernel Offset: disabled
[ 135.532496] ---[ end Kernel panic - not syncing: Fatal exception in interrupt
Thanks
Attachment:
signature.asc
Description: PGP signature
