> Il giorno 19 dic 2018, alle ore 04:45, Ming Lei <tom.leiming@xxxxxxxxx> ha scritto: > > On Wed, Dec 19, 2018 at 2:52 AM Jens Axboe <axboe@xxxxxxxxx> wrote: >> >> On 12/18/18 5:45 AM, Paolo Valente wrote: >>> Hi Jens, >>> sorry for the following silly question, but maybe you can solve very >>> quickly a doubt for which I'd spend much more time investigating. >>> >>> While doing some tests with scsi_debug, I've just seen that (at least) >>> with direct I/O, the maximum number of pending I/O requests (at least >>> in the I/O schedulers) is equal, unexpectedly, to the queue depth of >>> the drive and not to >>> /sys/block/<dev>/queue/nr_requests >>> >>> For example, after: >>> >>> sudo modprobe scsi_debug max_queue=4 >>> >>> and with fio executed as follows: >>> >>> job: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=20 >>> >>> I get this periodic trace, where four insertions are followed by four >>> completions, and so on, till the end of the I/O. This trace is taken >>> with none, but the result is the same with bfq. >>> >>> fio-20275 [001] d... 7560.655213: 8,48 I R 281088 + 8 [fio] >>> fio-20275 [001] d... 7560.655288: 8,48 I R 281096 + 8 [fio] >>> fio-20275 [001] d... 7560.655311: 8,48 I R 281104 + 8 [fio] >>> fio-20275 [001] d... 7560.655331: 8,48 I R 281112 + 8 [fio] >>> <idle>-0 [001] d.h. 7560.749868: 8,48 C R 281088 + 8 [0] >>> <idle>-0 [001] dNh. 7560.749912: 8,48 C R 281096 + 8 [0] >>> <idle>-0 [001] dNh. 7560.749928: 8,48 C R 281104 + 8 [0] >>> <idle>-0 [001] dNh. 7560.749934: 8,48 C R 281112 + 8 [0] >>> fio-20275 [001] d... 7560.750023: 8,48 I R 281120 + 8 [fio] >>> fio-20275 [001] d... 7560.750196: 8,48 I R 281128 + 8 [fio] >>> fio-20275 [001] d... 7560.750229: 8,48 I R 281136 + 8 [fio] >>> fio-20275 [001] d... 7560.750250: 8,48 I R 281144 + 8 [fio] >>> <idle>-0 [001] d.h. 7560.842510: 8,48 C R 281120 + 8 [0] >>> <idle>-0 [001] dNh. 7560.842551: 8,48 C R 281128 + 8 [0] >>> <idle>-0 [001] dNh. 7560.842556: 8,48 C R 281136 + 8 [0] >>> <idle>-0 [001] dNh. 7560.842562: 8,48 C R 281144 + 8 [0] >>> >>> Shouldn't the total number of pending requests reach >>> /sys/block/<dev>/queue/nr_requests ? >>> >>> The latter is of course equal to 8. >> >> With a scheduler, the depth is what the scheduler provides. You cannot >> exceed the hardware queue depth in any situation. You just have 8 >> requests available for scheduling, with a max of 4 being inflight on >> the device side. >> >> If both were 4, for instance, then you would have nothing to schedule >> with, as all of them could reside on the hardware side. That's why >> the scheduler defaults to twice the hardware queue depth. > > The default twice to hw queue depth might not be reasonable for multi LUN. > > Maybe it should be set twice of sdev->queue_depth for SCSI or > hw queue depth/hctx->nr_active. But either way may become complicated > because both can be adjusted runtime. > Could you please explain why it is not working (if it is not working) in my example, where there should be only one LUN? Thanks, Paolo > Thanks, > Ming Lei
