> -----Original Message----- > From: Christoph Hellwig [mailto:hch@xxxxxxxxxxxxx] > Sent: Thursday, 10 July, 2014 11:15 AM > To: Elliott, Robert (Server Storage) > Cc: Jens Axboe; dgilbert@xxxxxxxxxxxx; James Bottomley; Bart Van Assche; > Benjamin LaHaise; linux-scsi@xxxxxxxxxxxxxxx; linux-kernel@xxxxxxxxxxxxxxx > Subject: Re: scsi-mq V2 > > On Thu, Jul 10, 2014 at 09:04:22AM -0700, Christoph Hellwig wrote: > > It's starting to look weird. I'll prepare another two bisect branches > > around some MM changes, which seems the only other possible candidate. > > I've pushed out scsi-mq.3-bisect-3 Good. > and scsi-mq.3-bisect-4 for you. Bad. Note: I had to apply the vdso2c.h patch to build this -rc3 based kernel: diff --git a/arch/x86/vdso/vdso2c.h b/arch/x86/vdso/vdso2c.h index df95a2f..11b65d4 100644 --- a/arch/x86/vdso/vdso2c.h +++ b/arch/x86/vdso/vdso2c.h @@ -93,6 +93,9 @@ static void BITSFUNC(copy_section)(struct BITSFUNC(fake_sections) *out, uint64_t flags = GET_LE(&in->sh_flags); bool copy = flags & SHF_ALLOC && + (GET_LE(&in->sh_size) || + (GET_LE(&in->sh_type) != SHT_RELA && + GET_LE(&in->sh_type) != SHT_REL)) && strcmp(name, ".altinstructions") && strcmp(name, ".altinstr_replacement"); Results: fio started OK, getting 900K IOPS, but ^C led to 0 IOPS and an fio hang, with one CPU (CPU 0) stuck in io_submit loops. perf top shows lookup_ioctx function alongside io_submit and do_io_submit this time: 14.96% [kernel] [k] lookup_ioctx 14.71% libaio.so.1.0.1 [.] io_submit 13.78% [kernel] [k] system_call 10.79% [kernel] [k] system_call_after_swapgs 10.17% [kernel] [k] do_io_submit 8.91% [kernel] [k] copy_user_generic_string 4.24% [kernel] [k] io_submit_one 3.93% [kernel] [k] blk_flush_plug_list 3.32% fio [.] fio_libaio_commit 2.84% [kernel] [k] sysret_check 2.06% [kernel] [k] blk_finish_plug 1.89% [kernel] [k] SyS_io_submit 1.48% [kernel] [k] blk_start_plug 1.04% fio [.] io_submit@plt 0.84% [kernel] [k] __get_user_4 0.74% [kernel] [k] system_call_fastpath 0.60% [kernel] [k] _copy_from_user 0.51% diff [.] 0x0000000000007abb ftrace on CPU 0 shows similar repetition to before: fio-4107 [000] .... 389.992300: lookup_ioctx <-do_io_submit fio-4107 [000] .... 389.992300: blk_start_plug <-do_io_submit fio-4107 [000] .... 389.992300: io_submit_one <-do_io_submit fio-4107 [000] .... 389.992300: blk_finish_plug <-do_io_submit fio-4107 [000] .... 389.992300: blk_flush_plug_list <-blk_finish_plug fio-4107 [000] .... 389.992301: SyS_io_submit <-system_call_fastpath fio-4107 [000] .... 389.992301: do_io_submit <-SyS_io_submit fio-4107 [000] .... 389.992301: lookup_ioctx <-do_io_submit fio-4107 [000] .... 389.992301: blk_start_plug <-do_io_submit fio-4107 [000] .... 389.992301: io_submit_one <-do_io_submit fio-4107 [000] .... 389.992301: blk_finish_plug <-do_io_submit fio-4107 [000] .... 389.992301: blk_flush_plug_list <-blk_finish_plug fio-4107 [000] .... 389.992301: SyS_io_submit <-system_call_fastpath fio-4107 [000] .... 389.992302: do_io_submit <-SyS_io_submit fio-4107 [000] .... 389.992302: lookup_ioctx <-do_io_submit fio-4107 [000] .... 389.992302: blk_start_plug <-do_io_submit fio-4107 [000] .... 389.992302: io_submit_one <-do_io_submit fio-4107 [000] .... 389.992302: blk_finish_plug <-do_io_submit fio-4107 [000] .... 389.992302: blk_flush_plug_list <-blk_finish_plug -- To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
