[LSF/MM/BPF TOPIC] buffered IO atomic writes

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On v6.13 XFS got atomics write support through LBS. We have validated the
value of large atomics on databases and provided initial automation
for it through kdevops [0]. This requires direct IO today and there
are impressive confirmed results with it such as the ones Theodore Ts'o had
hinted at last year's LSFMM such as 3x-5x TPS variability gains. However the
results we have observed for buffered IO in PostgreSQL are even more
impressive: 14x-18x in TPS variability gains.

At least year's LSFMM we discused atomic buffered IO support, and if my
memory serves me correctly the conclusions where:

a) The PostgreSQL need for buffered IO due to lack of Direct IO is observed
   as a PostgreSQL mis-feature. So it is not a reason to add buffered IO
   atomic support

b) Near-writehrough buffered IO support would be good

c) Parallelizing writeback would be good

In so far as a) is concerned WiredTiger db is an example database which
although it supports both direct IO and buffered IO it strongly perfers
buffered IO. And so its an example of database which its users do
explicitly prefer buffered IO.

In so far as b) we now have RWF_DONTCACHE merged on v6.14-rc1. Will that
suffice? If not what are we missing?

And with regards to c) Kundan has suggested he's been working on parallelizing
writeback and its a sugested topic for LSFMM [1].

We have not re-tested PostgreSQL atomics benefits with RWF_DONTCACHE and
parallelizing writeback, however I suspect that may improve results even
further. So it seems to be a good time to ask, what else do we need for
buffered IO atomics?

[0] https://github.com/linux-kdevops/kdevops/blob/main/docs/sysbench/sysbench.md
[1] https://lore.kernel.org/all/20250129102627.161448-1-kundan.kumar@xxxxxxxxxxx/

  Luis




[Index of Archives]     [Linux Ext4 Filesystem]     [Union Filesystem]     [Filesystem Testing]     [Ceph Users]     [Ecryptfs]     [NTFS 3]     [AutoFS]     [Kernel Newbies]     [Share Photos]     [Security]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux Cachefs]     [Reiser Filesystem]     [Linux RAID]     [NTFS 3]     [Samba]     [Device Mapper]     [CEPH Development]

  Powered by Linux