On 2011-08-04 10:51, Martin Steigerwald wrote: > Am Mittwoch, 3. August 2011 schrieb Martin Steigerwald: >> Am Mittwoch, 3. August 2011 schrieb Martin Steigerwald: >>> Am Mittwoch, 3. August 2011 schrieben Sie: >>>> Martin Steigerwald <Martin@xxxxxxxxxxxx> writes: >> [...] >> >>> Does using iodepth > 1 need ioengine=libaio? Let´s see the manpage: >>> iodepth=int >>> >>> Number of I/O units to keep in flight against the >>> file. Note that increasing iodepth beyond 1 will >>> not affect synchronous ioengines (except for small >>> degress when verify_async is in use). Even async >>> engines my impose OS restrictions causing the >>> desired depth not to be achieved. This may happen >>> on Linux when using libaio and not setting >>> direct=1, since buffered IO is not async on that >>> OS. Keep an eye on the IO depth distribution in >>> the fio output to verify that the achieved depth >>> is as expected. Default: 1. >>> >>> Okay, yes, it does. I start getting a hang on it. Its a bit puzzling >>> to have two concepts of synchronous I/O around: >>> >>> 1) synchronous system call interfaces aka fio I/O engine >>> >>> 2) synchronous I/O requests aka O_SYNC >> >> But isn´t this a case for iodepth=1 if buffered I/O on Linux is >> synchronous? I bet most regular applications except some databases use >> buffered I/O. > > Thanks a lot for your answers, Jens, Jeff, DongJin. > > Now what about the above one? > > In what cases is iodepth > 1 relevant, when Linux buffered I/O is > synchronous? For mutiple threads or processes? iodepth controls what depth fio operates at, not the OS. You are right in that with iodepth=1, for buffered writes you could be seeing a much higher depth on the device side. So think of iodepth as how many IO units fio can have in flight, nothing else. > One process / thread can only submit one I/O at a time with synchronous > system call I/O, but the function returns when the stuff is in the page > cache. So first why can´t Linux use iodepth > 1 when there is lots of stuff > in the page cache to be written out? That should help the single process > case. Since the IO unit is done when the system call returns, you can never have more than the one in flight for a sync engine. So iodepth > 1 makes no sense for a sync engine. > On the mutiple process/threadsa case Linux gets several I/O requests from > mutiple processes/threads and thus iodepth > 1 does make sense? No. > Maybe it helps getting clear where in the stack iodepth is located at, is > it > > process / thread > systemcall > pagecache > blocklayer > iodepth > device driver > device > > ? If so, why can´t Linux not make use of iodepth > 1 with synchronous > system call I/O? Or is it further up on the system call level? But then Because it is sync. The very nature of the sync system calls is that submission and completion are one event. For libaio, you could submit a bunch of requests before retrieving or waiting for completion of any one of them. The only example where a sync engine could drive a higher queue depth on the device side is buffered writes. For any other case (reads, direct writes), you need async submission to build up a higher queue depth. > what sense would it make there, when using system calls that are > asynchronous already? > (Is that ordering above correct at all?) Your ordering looks OK. Now consider where and how you end up waiting for issued IO, that should tell you where queue depth could build up or not. -- Jens Axboe -- To unsubscribe from this list: send the line "unsubscribe fio" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
