On Tue, Aug 25, 2020 at 02:24:58PM +0200, Alberto Garcia wrote: > On Fri 21 Aug 2020 07:02:32 PM CEST, Brian Foster wrote: > >> I was running fio with --ramp_time=5 which ignores the first 5 seconds > >> of data in order to let performance settle, but if I remove that I can > >> see the effect more clearly. I can observe it with raw files (in 'off' > >> and 'prealloc' modes) and qcow2 files in 'prealloc' mode. With qcow2 and > >> preallocation=off the performance is stable during the whole test. > > > > That's interesting. I ran your fio command (without --ramp_time and > > with --runtime=5m) against a file on XFS (so no qcow2, no zero_range) > > once with sparse file with a 64k extent size hint and again with a > > fully preallocated 25GB file and I saw similar results in terms of the > > delta. This was just against an SSD backed vdisk in my local dev VM, > > but I saw ~5800 iops for the full preallocation test and ~6200 iops > > with the extent size hint. > > > > I do notice an initial iops burst as described for both tests, so I > > switched to use a 60s ramp time and 60s runtime. With that longer ramp > > up time, I see ~5000 iops with the 64k extent size hint and ~5500 iops > > with the full 25GB prealloc. Perhaps the unexpected performance delta > > with qcow2 is similarly transient towards the start of the test and > > the runtime is short enough that it skews the final results..? > > I also tried running directly against a file on xfs (no qcow2, no VMs) > but it doesn't really matter whether I use --ramp_time=5 or 60. > > Here are the results: > > |---------------+-------+-------| > | preallocation | xfs | ext4 | > |---------------+-------+-------| > | off | 7277 | 43260 | > | fallocate | 7299 | 42810 | > | full | 88404 | 83197 | > |---------------+-------+-------| > > I ran the first case (no preallocation) for 5 minutes and I said there's > a peak during the first 5 seconds, but then the number remains under 10k > IOPS for the rest of the 5 minutes. > I don't think we're talking about the same thing. I was referring to the difference between full file preallocation and the extent size hint in XFS, and how the latter was faster with the shorter ramp time but that swapped around when the test ramped up for longer. Here, it looks like you're comparing XFS to ext4 writing direct to a file.. If I compare this 5m fio test between XFS and ext4 on a couple of my systems (with either no prealloc or full file prealloc), I end up seeing ext4 run slightly faster on my vm and XFS slightly faster on bare metal. Either way, I don't see that huge disparity where ext4 is 5-6 times faster than XFS. Can you describe the test, filesystem and storage in detail where you observe such a discrepancy? Brian
