Re: io_submit with slab free object overwritten

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Reverted the commit 75374d062756 ("fs: add an iopoll method to struct
file_operations") fixed the problem. Christoph mentioned that the field can be
calculated by the offset (40 bytes).

struct kmem_cache {
        struct kmem_cache_cpu __percpu *cpu_slab; (8 bytes)
        slab_flags_t flags; (4)
        unsigned long min_partial; (8)
        unsigned int size; (4)
        unsigned int object_size; (4)
        unsigned int offset; (4)
        unsigned int cpu_partial; (4)
        struct kmem_cache_order_objects oo; (4)

        /* Allocation and freeing of slabs */
        struct kmem_cache_order_objects max;

So, it looks like "max" was overwritten after freed.

# cat /opt/ltp/runtest/syscalls
fgetxattr02 fgetxattr02
io_submit01 io_submit01

# /opt/ltp/runltp -f syscalls

uname:
Linux 5.0.0-rc7-next-20190222+ #11 SMP Fri Feb 22 14:57:10 EST 2019 ppc64le
ppc64le ppc64le GNU/Linux

/proc/cmdline
BOOT_IMAGE=/vmlinuz-5.0.0-rc7-next-20190222+
root=/dev/mapper/rhel_ibm--p8--01--lp5-root ro rd.lvm.lv=rhel_ibm-p8-01-lp5/root
rd.lvm.lv=rhel_ibm-p8-01-lp5/swap crashkernel=768M numa_balancing=enable earlyprintk

free reports:
              total        used        free      shared  buff/cache   available
Mem:       24305408      919552    23120832       12032      265024    22976896
Swap:       8388544           0     8388544

cpuinfo:
Architecture:        ppc64le
Byte Order:          Little Endian
CPU(s):              16
On-line CPU(s) list: 0-15
Thread(s) per core:  8
Core(s) per socket:  1
Socket(s):           2
NUMA node(s):        2
Model:               2.1 (pvr 004b 0201)
Model name:          POWER8 (architected), altivec supported
Hypervisor vendor:   pHyp
Virtualization type: para
L1d cache:           64K
L1i cache:           32K
L2 cache:            512K
L3 cache:            8192K
NUMA node0 CPU(s):
NUMA node1 CPU(s):   0-15

Running tests.......
<<<test_start>>>
tag=fgetxattr02 stime=1550865820
cmdline="fgetxattr02"
contacts=""
analysis=exit
<<<test_output>>>
tst_test.c:1096: INFO: Timeout per run is 0h 05m 00s
fgetxattr02.c:174: PASS: fgetxattr(2) on testfile passed
fgetxattr02.c:188: PASS: fgetxattr(2) on testfile got the right value
fgetxattr02.c:201: PASS: fgetxattr(2) on testfile passed: SUCCESS
fgetxattr02.c:174: PASS: fgetxattr(2) on testdir passed
fgetxattr02.c:188: PASS: fgetxattr(2) on testdir got the right value
fgetxattr02.c:201: PASS: fgetxattr(2) on testdir passed: SUCCESS
fgetxattr02.c:174: PASS: fgetxattr(2) on symlink passed
fgetxattr02.c:188: PASS: fgetxattr(2) on symlink got the right value
fgetxattr02.c:201: PASS: fgetxattr(2) on symlink passed: SUCCESS
fgetxattr02.c:201: PASS: fgetxattr(2) on fifo passed: ENODATA
fgetxattr02.c:201: PASS: fgetxattr(2) on chr passed: ENODATA
fgetxattr02.c:201: PASS: fgetxattr(2) on blk passed: ENODATA
fgetxattr02.c:201: PASS: fgetxattr(2) on sock passed: ENODATA

Summary:
passed   13
failed   0
skipped  0
warnings 0
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=0 cstime=1
<<<test_end>>>
<<<test_start>>>
tag=io_submit01 stime=1550865820
cmdline="io_submit01"
contacts=""
analysis=exit
<<<test_output>>>
incrementing stop
tst_test.c:1096: INFO: Timeout per run is 0h 05m 00s
io_submit01.c:125: PASS: io_submit() with invalid ctx failed with EINVAL
io_submit01.c:125: PASS: io_submit() with invalid nr failed with EINVAL
io_submit01.c:125: PASS: io_submit() with invalid iocbpp pointer failed with EFAULT
io_submit01.c:125: PASS: io_submit() with NULL iocb pointers failed with EFAULT
io_submit01.c:125: PASS: io_submit() with invalid fd failed with EBADF
io_submit01.c:125: PASS: io_submit() with readonly fd for write failed with EBADF
io_submit01.c:125: PASS: io_submit() with writeonly fd for read failed with EBADF
io_submit01.c:125: PASS: io_submit() with zero buf size failed with SUCCESS
io_submit01.c:125: PASS: io_submit() with zero nr failed with SUCCESS

Summary:
passed   9
failed   0
skipped  0
warnings 0

On 2/22/19 12:40 AM, Qian Cai wrote:
> This is only reproducible on linux-next (20190221), as v5.0-rc7 is fine. Running
> two LTP tests and then reboot will trigger this on ppc64le (CONFIG_IO_URING=n
> and CONFIG_SHUFFLE_PAGE_ALLOCATOR=y).
> 
> # fgetxattr02
> # io_submit01
> # systemctl reboot
> 
> There is a 32-bit (with all ones) overwritten of free slab objects (poisoned).
> 
> [23424.121182] BUG aio_kiocb (Tainted: G    B   W    L   ): Poison overwritten
> [23424.121189]
> -----------------------------------------------------------------------------
> [23424.121189]
> [23424.121197] INFO: 0x000000009f1f5145-0x00000000841e301b. First byte 0xff
> instead of 0x6b
> [23424.121205] INFO: Allocated in io_submit_one+0x9c/0xb20 age=0 cpu=7 pid=12174
> [23424.121212]  __slab_alloc+0x34/0x60
> [23424.121217]  kmem_cache_alloc+0x504/0x5c0
> [23424.121221]  io_submit_one+0x9c/0xb20
> [23424.121224]  sys_io_submit+0xe0/0x350
> [23424.121227]  system_call+0x5c/0x70
> [23424.121231] INFO: Freed in aio_complete+0x31c/0x410 age=0 cpu=7 pid=12174
> [23424.121234]  kmem_cache_free+0x4bc/0x540
> [23424.121237]  aio_complete+0x31c/0x410
> [23424.121240]  blkdev_bio_end_io+0x238/0x3e0
> [23424.121243]  bio_endio.part.3+0x214/0x330
> [23424.121247]  brd_make_request+0x2d8/0x314 [brd]
> [23424.121250]  generic_make_request+0x220/0x510
> [23424.121254]  submit_bio+0xc8/0x1f0
> [23424.121256]  blkdev_direct_IO+0x36c/0x610
> [23424.121260]  generic_file_read_iter+0xbc/0x230
> [23424.121263]  blkdev_read_iter+0x50/0x80
> [23424.121266]  aio_read+0x138/0x200
> [23424.121269]  io_submit_one+0x7c4/0xb20
> [23424.121272]  sys_io_submit+0xe0/0x350
> [23424.121275]  system_call+0x5c/0x70
> [23424.121278] INFO: Slab 0x00000000841158ec objects=85 used=85 fp=0x
> (null) flags=0x13fffc000000200
> [23424.121282] INFO: Object 0x000000007e677ed8 @offset=5504 fp=0x00000000e42bdf6f
> [23424.121282]
> [23424.121287] Redzone 000000005483b8fc: bb bb bb bb bb bb bb bb bb bb bb bb bb
> bb bb bb  ................
> [23424.121291] Redzone 00000000b842fe53: bb bb bb bb bb bb bb bb bb bb bb bb bb
> bb bb bb  ................
> [23424.121295] Redzone 00000000deb0d052: bb bb bb bb bb bb bb bb bb bb bb bb bb
> bb bb bb  ................
> [23424.121299] Redzone 0000000014045233: bb bb bb bb bb bb bb bb bb bb bb bb bb
> bb bb bb  ................
> [23424.121302] Redzone 00000000dd5d6c16: bb bb bb bb bb bb bb bb bb bb bb bb bb
> bb bb bb  ................
> [23424.121306] Redzone 00000000538b5478: bb bb bb bb bb bb bb bb bb bb bb bb bb
> bb bb bb  ................
> [23424.121310] Redzone 000000001f7fb704: bb bb bb bb bb bb bb bb bb bb bb bb bb
> bb bb bb  ................
> [23424.121314] Redzone 0000000000e0484d: bb bb bb bb bb bb bb bb bb bb bb bb bb
> bb bb bb  ................
> [23424.121318] Object 000000007e677ed8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
> 6b 6b 6b  kkkkkkkkkkkkkkkk
> [23424.121322] Object 00000000e207f30b: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
> 6b 6b 6b  kkkkkkkkkkkkkkkk
> [23424.121326] Object 00000000a7a45634: 6b 6b 6b 6b 6b 6b 6b 6b ff ff ff ff 6b
> 6b 6b 6b  kkkkkkkk....kkkk
> [23424.121330] Object 00000000c85d951d: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
> 6b 6b 6b  kkkkkkkkkkkkkkkk
> [23424.121334] Object 000000003104522f: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
> 6b 6b 6b  kkkkkkkkkkkkkkkk
> [23424.121338] Object 00000000cfcdd820: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
> 6b 6b 6b  kkkkkkkkkkkkkkkk
> [23424.121342] Object 00000000dded4924: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
> 6b 6b 6b  kkkkkkkkkkkkkkkk
> [23424.121346] Object 00000000ff6687a4: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
> 6b 6b 6b  kkkkkkkkkkkkkkkk
> [23424.121350] Object 00000000df3d67f6: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
> 6b 6b 6b  kkkkkkkkkkkkkkkk
> [23424.121354] Object 00000000ddc188d1: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
> 6b 6b 6b  kkkkkkkkkkkkkkkk
> [23424.121358] Object 000000002cee751a: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
> 6b 6b 6b  kkkkkkkkkkkkkkkk
> [23424.121362] Object 00000000a994f007: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
> 6b 6b a5  kkkkkkkkkkkkkkk.
> [23424.121366] Redzone 000000009f3d62e2: bb bb bb bb bb bb bb bb
>          ........
> [23424.121370] Padding 00000000e5ccead8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a
> 5a 5a 5a  ZZZZZZZZZZZZZZZZ
> [23424.121374] Padding 000000002b0c1778: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a
> 5a 5a 5a  ZZZZZZZZZZZZZZZZ
> [23424.121378] Padding 00000000c67656c7: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a
> 5a 5a 5a  ZZZZZZZZZZZZZZZZ
> [23424.121382] Padding 0000000078348c5a: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a
> 5a 5a 5a  ZZZZZZZZZZZZZZZZ
> [23424.121386] Padding 00000000f3297820: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a
> 5a 5a 5a  ZZZZZZZZZZZZZZZZ
> [23424.121390] Padding 00000000e55789f4: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a
> 5a 5a 5a  ZZZZZZZZZZZZZZZZ
> [23424.121394] Padding 00000000d0fbb94c: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a
> 5a 5a 5a  ZZZZZZZZZZZZZZZZ
> [23424.121397] Padding 00000000bcb27a87: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a
> 5a 5a 5a  ZZZZZZZZZZZZZZZZ
> [23424.121743] CPU: 7 PID: 12174 Comm: vgs Tainted: G    B   W    L
> 5.0.0-rc7-next-20190221+ #7
> [23424.121758] Call Trace:
> [23424.121762] [c0000004ce5bf7b0] [c0000000007deb8c] dump_stack+0xb0/0xf4
> (unreliable)
> [23424.121770] [c0000004ce5bf7f0] [c00000000037d310] print_trailer+0x250/0x278
> [23424.121775] [c0000004ce5bf880] [c00000000036d578]
> check_bytes_and_report+0x138/0x160
> [23424.121779] [c0000004ce5bf920] [c00000000036fac8] check_object+0x348/0x3e0
> [23424.121784] [c0000004ce5bf990] [c00000000036fd18]
> alloc_debug_processing+0x1b8/0x2c0
> [23424.121788] [c0000004ce5bfa30] [c000000000372d14] ___slab_alloc+0xbb4/0xfa0
> [23424.121792] [c0000004ce5bfb60] [c000000000373134] __slab_alloc+0x34/0x60
> [23424.121802] [c0000004ce5bfb90] [c000000000373664] kmem_cache_alloc+0x504/0x5c0
> [23424.121812] [c0000004ce5bfc20] [c000000000476a9c] io_submit_one+0x9c/0xb20
> [23424.121824] [c0000004ce5bfd50] [c000000000477f10] sys_io_submit+0xe0/0x350
> [23424.121832] [c0000004ce5bfe20] [c00000000000b000] system_call+0x5c/0x70
> [23424.121836] FIX aio_kiocb: Restoring 0x000000009f1f5145-0x00000000841e301b=0x6b
> [23424.121836]
> [23424.121840] FIX aio_kiocb: Marking all objects used
> 




[Index of Archives]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux