> > On 12/01/2021 11:44, Kashyap Desai wrote: > >>>> loops = 10000 > >>> Is there any effect on random read IOPS when you decrease sdev queue > >>> depth? For sequential IO, IO merge can be enhanced by that way. > >>> > >> Let me check... > > John - Can you check your test with rq_affinity = 2 and nomerges=2 ? > > > > I have noticed similar drops (whatever you have reported) - "once we > > reach near pick of sdev or host queue depth, sometimes performance > > drop due to contention." > > But this behavior keep changing since kernel changes in this area is > > very actively happened in past. So I don't know the exact details > > about kernel version etc. > > I have similar setup (16 SSDs) and I will try similar test on latest > > kernel. > > > > BTW - I remember that rq_affinity=2 play major role in such issue. I > > usually do testing with rq_affinity = 2. > > > > Hi Kashyap, > > As requested: > > rq_affinity=1, nomerges=0 (default) > > sdev queue depth num jobs=1 > 8 1650 > 16 1638 > 32 1612 > 64 1573 > 254 1435 (default for LLDD) > > rq_affinity=2, nomerges=2 > > sdev queue depth num jobs=1 > 8 1236 > 16 1423 > 32 1438 > 64 1438 > 254 1438 (default for LLDD) > > Setup as original: fio read, 12x SAS SSDs What is an issue with rq_affinity=2 and nomerges=2 ? It looks like - "Dropping IOPs from peak (1.6M) is not an issue here but we are not able to reach peak IOPs. IOPS increase gradually and it saturate." > > So, again, we see that performance changes from changing sdev queue depth > depends on workload and then also other queue config. Can you send me "dmesg" logs of your setup. What is host_busy counter while you run your test ? I am not able to duplicate the issue on my setup. To avoid any io merge issue, can we use rand_read data ? Kashyap > > Thanks, > John -- This electronic communication and the information and any files transmitted with it, or attached to it, are confidential and are intended solely for the use of the individual or entity to whom it is addressed and may contain information that is confidential, legally privileged, protected by privacy laws, or otherwise restricted from disclosure to anyone else. If you are not the intended recipient or the person responsible for delivering the e-mail to the intended recipient, you are hereby notified that any use, copying, distributing, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited. If you received this e-mail in error, please return the e-mail to the sender, delete it from your computer, and destroy any printed copy of it.
Attachment:
smime.p7s
Description: S/MIME Cryptographic Signature
