On Thu, Jan 24, 2019 at 02:33:19PM +0200, Leon Romanovsky wrote:
> From: Feras Daoud <ferasda@xxxxxxxxxxxx>
>
> The following BUG was reported by kasan:
>
> ==================================================================
> [27921.899778] BUG: KASAN: use-after-free in ipoib_cm_tx_start+0x430/0x1390 [ib_ipoib]
> [27921.899954] Read of size 80 at addr ffff88034c30bcd0 by task kworker/u16:1/24020
>
> [27921.900313] Workqueue: ipoib_wq ipoib_cm_tx_start [ib_ipoib]
> [27921.900324] Call Trace:
> [27921.900340] dump_stack+0x9a/0xeb
> [27921.900369] print_address_description+0xe3/0x2e0
> [27921.900393] kasan_report+0x18a/0x2e0
> [27921.900408] ? ipoib_cm_tx_start+0x430/0x1390 [ib_ipoib]
> [27921.900430] memcpy+0x1f/0x50
> [27921.900449] ipoib_cm_tx_start+0x430/0x1390 [ib_ipoib]
> [27921.900473] ? kvm_clock_read+0x1f/0x30
> [27921.900510] ? ipoib_cm_skb_reap+0x610/0x610 [ib_ipoib]
> [27921.900535] ? __lock_is_held+0xc2/0x170
> [27921.900547] ? process_one_work+0x880/0x1960
> [27921.900578] ? process_one_work+0x912/0x1960
> [27921.900585] process_one_work+0x912/0x1960
> [27921.900622] ? wq_pool_ids_show+0x310/0x310
> [27921.900630] ? lock_acquire+0x145/0x440
> [27921.900682] worker_thread+0x87/0xbb0
> [27921.900723] ? process_one_work+0x1960/0x1960
> [27921.900733] kthread+0x314/0x3d0
> [27921.900742] ? kthread_create_worker_on_cpu+0xc0/0xc0
> [27921.900760] ret_from_fork+0x3a/0x50
>
> [27921.900843] Allocated by task 0:
> [27921.900919] kasan_kmalloc+0xa0/0xd0
> [27921.900925] kmem_cache_alloc_trace+0x168/0x3e0
> [27921.900937] path_rec_create+0xa2/0x1f0 [ib_ipoib]
> [27921.900947] ipoib_start_xmit+0xa98/0x19e0 [ib_ipoib]
> [27921.900955] dev_hard_start_xmit+0x159/0x8d0
> [27921.900963] sch_direct_xmit+0x226/0xb40
> [27921.900969] __dev_queue_xmit+0x1d63/0x2950
> [27921.900976] neigh_update+0x889/0x1770
> [27921.900983] arp_process+0xc47/0x21f0
> [27921.900988] arp_rcv+0x462/0x760
> [27921.900995] __netif_receive_skb_core+0x1546/0x2da0
> [27921.901002] netif_receive_skb_internal+0xf2/0x590
> [27921.901023] napi_gro_receive+0x28e/0x390
> [27921.901045] ipoib_ib_handle_rx_wc_rss+0x873/0x1b60 [ib_ipoib]
> [27921.901065] ipoib_rx_poll_rss+0x17d/0x320 [ib_ipoib]
> [27921.901074] net_rx_action+0x427/0xe30
> [27921.901081] __do_softirq+0x28e/0xc42
>
> [27921.901128] Freed by task 26680:
> [27921.901211] __kasan_slab_free+0x11d/0x160
> [27921.901218] kfree+0xf5/0x360
> [27921.901242] ipoib_flush_paths+0x532/0x9d0 [ib_ipoib]
> [27921.901255] ipoib_set_mode_rss+0x1ad/0x560 [ib_ipoib]
> [27921.901268] set_mode+0xc8/0x150 [ib_ipoib]
> [27921.901278] kernfs_fop_write+0x279/0x440
> [27921.901286] __vfs_write+0xd8/0x5c0
> [27921.901293] vfs_write+0x15e/0x470
> [27921.901300] ksys_write+0xb8/0x180
> [27921.901308] do_syscall_64+0x9b/0x420
> [27921.901316] entry_SYSCALL_64_after_hwframe+0x49/0xbe
>
> [27921.901367] The buggy address belongs to the object at ffff88034c30bcc8
> which belongs to the cache kmalloc-512 of size 512
> [27921.901617] The buggy address is located 8 bytes inside of
> 512-byte region [ffff88034c30bcc8, ffff88034c30bec8)
> [27921.901839] The buggy address belongs to the page:
> [27921.901941] page:ffffea000d30c200 count:1 mapcount:0 mapping:ffff8803554111c0 index:0xffff88034c30b968 compound_mapcount: 0
> [27921.902317] flags: 0x2fffff80008100(slab|head)
> [27921.902468] raw: 002fffff80008100 ffff880355403450 ffffea000d194a08 ffff8803554111c0
> [27921.902595] raw: ffff88034c30b968 000000000025000b 00000001ffffffff 0000000000000000
> [27921.902719] page dumped because: kasan: bad access detected
>
> [27921.902850] Memory state around the buggy address:
> [27921.902934] ffff88034c30bb80: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
> [27921.903096] ffff88034c30bc00: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
> [27921.903237] >ffff88034c30bc80: fc fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb
> [27921.903376] ^
> [27921.903492] ffff88034c30bd00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> [27921.903633] ffff88034c30bd80: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> ==================================================================
>
> The following race between change mode and xmit flow is the reason for
> this use-after-free:
>
> Change mode Send packet 1 to GID XX Send packet 2 to GID XX
> | | |
> start | |
> | | |
> | | |
> | Create new path for GID XX |
> | and update neigh path |
> | | |
> | | |
> | | |
> flush_paths | |
> | |
> queue_work(cm.start_task) |
> | Path for GID XX not found
> | create new path
> |
> |
> start_task runs with old
> released path
>
> To fix this issue, use the new path that __path_find returns.
>
> Fixes: 546481c2816e ("IB/ipoib: Fix memory corruption in ipoib cm mode connect flow")
> Signed-off-by: Feras Daoud <ferasda@xxxxxxxxxxxx>
> Reviewed-by: Erez Shitrit <erezsh@xxxxxxxxxxxx>
> Signed-off-by: Leon Romanovsky <leonro@xxxxxxxxxxxx>
> ---
> drivers/infiniband/ulp/ipoib/ipoib_cm.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
Applied to for-rc, with Ira's note added and reworked commmit message
explaining the path deletion.
Jason
