On Tue, Sep 10, 2024 at 06:11:50PM -0400, Mike Snitzer wrote: > On Tue, Sep 10, 2024 at 04:31:23PM -0400, Anna Schumaker wrote: > > Hi Mike, > > > > On 9/10/24 3:14 PM, Mike Snitzer wrote: > > > > > > In case it helps, I did just rebase LOCALIO (v16 + 1 fix) ontop of > > > cel/nfsd-next (v6.11-rc6 based), and I've pushed the result here: > > > https://git.kernel.org/pub/scm/linux/kernel/git/snitzer/linux.git/log/?h=nfs-localio-for-next > > > > I'm seeing the same hang on generic/525 with your latest branch. > > > > Anna > > Interesting, I just looked at ktest and it shows the regression point > to be this commit: > nfs: implement client support for NFS_LOCALIO_PROGRAM > > See: > https://evilpiepirate.org/~testdashboard/ci?user=snitzer&branch=snitm-nfs-next&test=^fs.nfs.fstests.generic.525$ > > I think 525 has been like this for a while, really not sure why I > ignored it... will dig deeper! I haven't found/fixed this yet but when LOCALIO is used xfs_file_buffered_read calls filemap_get_pages and filemap_get_pages livelocks trying to complete. Here is ftrace of LOCALIO's xfs_file_buffered_read immediately before filemap_get_pages never returns (note that for debugging I reverted the workqueue patch.. so the IO is issued in xfs_io context): => entry_SYSCALL_64_after_hwframe xfs_io-5609 [007] ..... 211.831093: xfs_file_buffered_read: dev 8:16 ino 0x84 disize 0x7fffffffffffffff pos 0x7ffffffffffff000 bytecount 0x1000 xfs_io-5609 [007] ...1. 211.831098: <stack trace> => trace_event_raw_event_xfs_file_class => xfs_file_buffered_read => xfs_file_read_iter => nfs_local_doio => nfs_initiate_pgio => nfs_generic_pg_pgios => nfs_pageio_doio => nfs_pageio_complete => nfs_pageio_complete_read => nfs_readahead => read_pages => page_cache_ra_unbounded => page_cache_sync_ra => filemap_get_pages => filemap_read => generic_file_read_iter => nfs_file_read => vfs_read => __x64_sys_pread64 => x64_sys_call => do_syscall_64 Here is the same when testing only against XFS: => entry_SYSCALL_64_after_hwframe xfs_io-3451 [015] ..... 1034.767416: xfs_file_buffered_read: dev 8:16 ino 0x84 disize 0x7fffffffffffffff pos 0x7ffffffffffffffe bytecount 0x1 xfs_io-3451 [015] ...1. 1034.767418: <stack trace> => trace_event_raw_event_xfs_file_class => xfs_file_buffered_read => xfs_file_read_iter => vfs_read => __x64_sys_pread64 => x64_sys_call => do_syscall_64 Will keep after this with urgency, just wanted to let you know what I have found so far... Thanks, Mike
