> Il giorno 9 nov 2018, alle ore 21:21, Elliott, Robert (Persistent Memory) <elliott@xxxxxxx> ha scritto: > > > >> -----Original Message----- >> From: Federico Motta [mailto:federico@xxxxxxxxx] >> Sent: Friday, November 9, 2018 3:31 AM >> To: Sitsofe Wheeler <sitsofe@xxxxxxxxx> >> Cc: Jeff Furlong <jeff.furlong@xxxxxxx>; Elliott, Robert (Persistent Memory) <elliott@xxxxxxx>; fio >> <fio@xxxxxxxxxxxxxxx>; Paolo Valente <paolo.valente@xxxxxxxxxx>; mark.brown@xxxxxxxxxx; >> linus.walleij@xxxxxxxxxx; ulf.hansson@xxxxxxxxxx >> Subject: Re: rand-read: increasing iodepth makes throughput drop >>>> On 10/31/18 3:19 PM, Federico Motta wrote: >>>> ... >>>>> Is it possible to ask fio a measure of "how much device internal >>>> queues are filled"? >>>>> If yes, with which option(s)? >>>>> If not, do you know some other tools or procedures which can tell us that? >>>> >>>> Does somebody has ideas about that ^ ? >>> >>> ...I think iostat produces similar output to that too... >>> >> >> Actually I was wondering if a more direct or low level measure exists, >> but since I don't even know if that is possible I think the fio "util" >> percentage could be a good trade-off between "a low level measure" and >> "no measure available". > > In linux, iostat provides some statistics, and there are lots of more detailed > queuing statistics in sysfs. Some are provided by the block layer; some by the > SCSI midlayer (for SCSI devices); some by the low-level driver (like hpsa). > > > # nr_requests, the block layer request queue depth > /sys/block/$device/queue/nr_requests > > # queue_depth, the SCSI midlayer queue depth > /sys/block/$device/device/queue_depth > > # can_queue sets the initial request queue size > /sys/class/scsi_host/$scsi_host/can_queue > > # cmd_per_lun sets the initial queue_depth > /sys/class/scsi_host/$scsi_host/cmd_per_lun > > # host_busy indicates how many commands are queued by the SCSI layer > /sys/class/scsi_host/$scsi_host/host_busy > Thank you very much, this seems exactly the kind of information we need. AFAYK, are there also per-queue info in case of NVMe devices? Thanks, Paolo > # host_blocked indicates the host is blocking all IOs > /sys/class/scsi_host/$scsi_host/host_blocked > > # hpsa ioaccel commands outstanding > /sys/block/$device/device/hpsa_ioaccel_cmds_out > > # hpsa physical drive queue depth > /sys/block/$device/device/hpsa_queue_depth > > # hpsa commands_outstanding from submission to completion > /sys/class/scsi_host/$scsi_host/commands_outstanding > > There are a bunch of blk-mq stats too, some only shown if debugfs is enabled. >
