On Tue, Feb 04, 2025 at 11:40:00AM -0700, Jens Axboe wrote:
> Read side was already fully supported, and with the write side
> appropriately punted to the worker queue, all that's needed now is
> setting FOP_DONTCACHE in the file_operations structure to enable full
> support for read and write uncached IO.
>
> This provides similar benefits to using RWF_DONTCACHE with reads. Testing
> buffered writes on 32 files:
>
> writing bs 65536, uncached 0
> 1s: 196035MB/sec
> 2s: 132308MB/sec
> 3s: 132438MB/sec
> 4s: 116528MB/sec
> 5s: 103898MB/sec
> 6s: 108893MB/sec
> 7s: 99678MB/sec
> 8s: 106545MB/sec
> 9s: 106826MB/sec
> 10s: 101544MB/sec
> 11s: 111044MB/sec
> 12s: 124257MB/sec
> 13s: 116031MB/sec
> 14s: 114540MB/sec
> 15s: 115011MB/sec
> 16s: 115260MB/sec
> 17s: 116068MB/sec
> 18s: 116096MB/sec
>
> where it's quite obvious where the page cache filled, and performance
> dropped from to about half of where it started, settling in at around
> 115GB/sec. Meanwhile, 32 kswapds were running full steam trying to
> reclaim pages.
>
> Running the same test with uncached buffered writes:
>
> writing bs 65536, uncached 1
> 1s: 198974MB/sec
> 2s: 189618MB/sec
> 3s: 193601MB/sec
> 4s: 188582MB/sec
> 5s: 193487MB/sec
> 6s: 188341MB/sec
> 7s: 194325MB/sec
> 8s: 188114MB/sec
> 9s: 192740MB/sec
> 10s: 189206MB/sec
> 11s: 193442MB/sec
> 12s: 189659MB/sec
> 13s: 191732MB/sec
> 14s: 190701MB/sec
> 15s: 191789MB/sec
> 16s: 191259MB/sec
> 17s: 190613MB/sec
> 18s: 191951MB/sec
>
> and the behavior is fully predictable, performing the same throughout
> even after the page cache would otherwise have fully filled with dirty
> data. It's also about 65% faster, and using half the CPU of the system
> compared to the normal buffered write.
>
> Signed-off-by: Jens Axboe <axboe@xxxxxxxxx>
Reviewed-by: "Darrick J. Wong" <djwong@xxxxxxxxxx>
--D
> ---
> fs/xfs/xfs_file.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
> index f7a7d89c345e..358987b6e2f8 100644
> --- a/fs/xfs/xfs_file.c
> +++ b/fs/xfs/xfs_file.c
> @@ -1626,7 +1626,8 @@ const struct file_operations xfs_file_operations = {
> .fadvise = xfs_file_fadvise,
> .remap_file_range = xfs_file_remap_range,
> .fop_flags = FOP_MMAP_SYNC | FOP_BUFFER_RASYNC |
> - FOP_BUFFER_WASYNC | FOP_DIO_PARALLEL_WRITE,
> + FOP_BUFFER_WASYNC | FOP_DIO_PARALLEL_WRITE |
> + FOP_DONTCACHE,
> };
>
> const struct file_operations xfs_dir_file_operations = {
> --
> 2.47.2
>
>
