On Sun, Jan 12, 2025 at 7:45 AM lizetao <lizetao1@xxxxxxxxxx> wrote:
>
> Hi,
>
> > -----Original Message-----
> > From: Jens Axboe <axboe@xxxxxxxxx>
> > Sent: Sunday, January 12, 2025 1:13 AM
> > To: lizetao <lizetao1@xxxxxxxxxx>; io-uring <io-uring@xxxxxxxxxxxxxxx>
> > Cc: Pavel Begunkov <asml.silence@xxxxxxxxx>
> > Subject: Re: KASAN reported an error while executing accept-reust.t testcase
> >
> > On 1/11/25 7:07 AM, lizetao wrote:
> > > Hi all,
> > >
> > > When I run the testcase liburing/accept-reust.t with CONFIG_KASAN=y
> > > and CONFIG_KASAN_EXTRA_INFO=y, I got a error reported by KASAN:
> >
> > Looks more like you get KASAN crashing...
> >
> > > Unable to handle kernel paging request at virtual address
> > > 00000c6455008008 Mem abort info:
> > > ESR = 0x0000000096000004
> > > EC = 0x25: DABT (current EL), IL = 32 bits
> > > SET = 0, FnV = 0
> > > EA = 0, S1PTW = 0
> > > FSC = 0x04: level 0 translation fault Data abort info:
> > > ISV = 0, ISS = 0x00000004, ISS2 = 0x00000000
> > > CM = 0, WnR = 0, TnD = 0, TagAccess = 0
> > > GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0 user pgtable: 4k pages,
> > > 48-bit VAs, pgdp=00000001104c5000 [00000c6455008008]
> > > pgd=0000000000000000, p4d=0000000000000000 Internal error: Oops:
> > > 0000000096000004 [#1] PREEMPT SMP Modules linked in:
> > > CPU: 6 UID: 0 PID: 352 Comm: kworker/u128:5 Not tainted
> > > 6.13.0-rc6-g0a2cb793507d #5 Hardware name: linux,dummy-virt (DT)
> > > Workqueue: iou_exit io_ring_exit_work
> > > pstate: 10000005 (nzcV daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) pc :
> > > __kasan_mempool_unpoison_object+0x38/0x170
> > > lr : io_netmsg_cache_free+0x8c/0x180
> > > sp : ffff800083297a90
> > > x29: ffff800083297a90 x28: ffffd4d7f67e88e4 x27: 0000000000000003
> > > x26: 1fffe5958011502e x25: ffff2cabff976c18 x24: 1fffe5957ff2ed83
> > > x23: ffff2cabff976c10 x22: 00000c6455008000 x21: 0002992540200001
> > > x20: 0000000000000000 x19: 00000c6455008000 x18: 00000000489683f8
> > > x17: ffffd4d7f68006ac x16: ffffd4d7f67eb3e0 x15: ffffd4d7f67e88e4
> > > x14: ffffd4d7f766deac x13: ffffd4d7f6619030 x12: ffff7a9b012e3e26
> > > x11: 1ffffa9b012e3e25 x10: ffff7a9b012e3e25 x9 : ffffd4d7f766debc
> > > x8 : ffffd4d80971f128 x7 : 0000000000000001 x6 : 00008564fed1c1db
> > > x5 : ffffd4d80971f128 x4 : ffff7a9b012e3e26 x3 : ffff2cabff976c00
> > > x2 : ffffc1ffc0000000 x1 : 0000000000000000 x0 : 0002992540200001 Call
> > > trace:
> > > __kasan_mempool_unpoison_object+0x38/0x170 (P)
> > > io_netmsg_cache_free+0x8c/0x180
> > > io_ring_exit_work+0xd4c/0x13a0
> > > process_one_work+0x52c/0x1000
> > > worker_thread+0x830/0xdc0
> > > kthread+0x2bc/0x348
> > > ret_from_fork+0x10/0x20
> > > Code: aa0003f5 aa0103f4 8b131853 aa1303f6 (f9400662) ---[ end trace
> > > 0000000000000000 ]---
> > >
> > >
> > > I preliminary analyzed the accept and connect code logic. In the
> > > accept-reuse.t testcase, kmsg->free_iov is not used, so when calling
> > > io_netmsg_cache_free(), the
> > > kasan_mempool_unpoison_object(kmsg->free_iov...) path should not be
> > > executed.
> > >
> > >
> > > I used the hardware watchpoint to capture the first scene of modifying kmsg-
> > >free_iov:
> > >
> > > Thread 3 hit Hardware watchpoint 7: *0xffff0000ebfc5410 Old value = 0
> > > New value = -211812350 kasan_set_track (stack=<optimized out>,
> > > track=<optimized out>) at ./arch/arm64/include/asm/current.h:21
> > > 21 return (struct task_struct *)sp_el0;
> > >
> > > # bt
> > > kasan_set_track
> > > kasan_save_track
> > > kasan_save_free_info
> > > poison_slab_object
> > > __kasan_mempool_poison_object
> > > kasan_mempool_poison_object
> > > io_alloc_cache_put
> > > io_netmsg_recycle
> > > io_req_msg_cleanup
> > > io_connect
> > > io_issue_sqe
> > > io_queue_sqe
> > > io_req_task_submit
> > > ...
> > >
> > >
> > > It's a bit strange. It was modified by KASAN. I can't understand this.
> > > Maybe I missed something? Please let me know. Thanks.
> >
> > Looks like KASAN with the extra info ends up writing to io_async_msghdr-
> > >free_iov somehow. No idea... For the test case in question, ->free_iov should
> > be NULL when initially allocated, and the io_uring code isn't storing to it. Yet
> > it's non-NULL when you later go and free it, after calling
> > kasan_mempool_poison_object().
>
> I also think so and would Juntong and Ryabinin or others KASAN developers be interested
> In this problem?
>
Hi, thanks for reporting.
KASAN stores some info about freed slab object in the object itself
until it is reallocated or the slab page is released.
And since the b556a462eb8d ("kasan: save free stack traces for slab
mempools") we do the same thing in kasan_mempool_poison_object().
In the most use cases this wasn't the problem, because callers expect
uninitialized objects from mempool.
However, this isn't the case for io_alloc_cache. AFAICS io_uring code
expects that io_alloc_cache_put/get leaves objects unmodified.
So I'm thinking we'd need to add some parameter to the
kasan_mempool_poison_object() to avoid modifying objects.
