Hello Dave, On 01.02.2016 06:46, Dave Chinner wrote: > On Sat, Jan 30, 2016 at 11:43:56AM +0100, Christian Affolter wrote: >> Hi Dave, >> >> On 29.01.2016 23:25, Dave Chinner wrote: >>> On Fri, Jan 29, 2016 at 11:53:35AM +0100, Christian Affolter wrote: >>>> Hi everyone, >>>> >>>> I'm trying to understand the differences of some bandwidth and IOPs test >>>> results I see while running a random-write full-stripe-width aligned fio >>>> test (using libaio with direct IO) on a hardware RAID 6 raw device >>>> versus on the same device with the XFS file system on top of it. >>>> >>>> On the raw device I get: >>>> write: io=24828MB, bw=423132KB/s, iops=137, runt= 60085msec >>>> >>>> With XFS on top of it: >>>> write: io=14658MB, bw=249407KB/s, iops=81, runt= 60182msec >>> >>> Now repeat with a file that is contiguously allocated before you >>> start. And also perhaps with the "swalloc" mount option. >> >> Wow, thanks! After specifying --fallocate=none (instead of the default >> fallocate=posix), bandwidth and iops increases and are even higher than >> on the raw device: >> >> write: io=30720MB, bw=599232KB/s, iops=195, runt= 52496msec >> >> I'm eager to learn what's going on behind the scenes, can you give a >> short explanation? > > Usually when concurrent direct IO writes are slower than the raw > device it's because something is causing IO submission > serialisation. Usually that's to do with writes that extend the > file because that can require the inode to be locked exclusively. > Whatever behaviour the fio configuration change modifed, it removed > the IO submission serialisation and so it's now running at full disk > speed. > > As to why XFS is faster than the raw block device, the XFS file > is only 30GB, so the random writes are only seeking a short > distance compared to the block device test which is seeking across > the whole device. > >> Btw. mounting the volume with "swalloc" didn't make any change. > > Which means there is no performance differential between stripe unit > and stripe width aligned writes in this test on your hardware. Thank you so much for the detailed explanation and taking the time to help. Best, Chris _______________________________________________ xfs mailing list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
