> -----Original Message----- > From: fio-owner@xxxxxxxxxxxxxxx <fio-owner@xxxxxxxxxxxxxxx> On Behalf > Of Elliott Balsley > Sent: Thursday, April 30, 2020 3:52 PM > To: fio <fio@xxxxxxxxxxxxxxx> > Subject: Monitoring disk i/o in Windows > > I built a Windows 2019 server with an NVMe storage pool and I'm using > fio to test performance. Results from fio are in line with my > expectations for this hardware, but I noticed something strange with > Windows Performance Monitor. The physical disk r/w throughput > counters are roughly double what they should be. This holds true for > various different iodepth, numjobs, and block size options. Logical > Disk throughput counters are accurate. In this screenshot, fio shows > 11.1GiB/s and the physical disk shows about 22.2 GiB/s. Is there a > logical explanation for this? Or is there a better disk monitoring > tool to use, something like iostat for Windows? > > https://i.postimg.cc/zqrW8ZPk/windows-fio.png?dl=1 > > The storage pool is created as a simple stripe with no parity or > mirroring and formatted as ReFS. Writing to files means you're subject to however the filesystem works. Maybe it's writing everything twice as part of journaling? Accessing the raw physical device would be simpler. It looks like fio's 1 MiB IOs are getting split into smaller IOs. Although your LogicalDisk Avg Disk Bytes/Write is 1 MiB, the PhysicalDisk reports 419,422.101 bytes (which is just below 400 KiB). If not just fundamental to the filesystem, it might be exceeding a limit somewhere in the stack (e.g., too many scatter-gather list entries or PRP entries). ~10 years ago we ran into a limit somewhere in the stack of 255 max SG entries, with each entry describing one 4 KiB CPU page, which was 4 KiB short of a 1 MiB transfer size. That kept 1 MiB IOs from going through in Windows.
