Steve French <smfrench@xxxxxxxxx> wrote:
> This (readahead behavior improvements in Linux, on single large file
> sequential read workloads like cp or grep) gets particularly interesting
> with SMB3 as multichannel becomes more common. With one channel having one
> readahead request pending on the network is suboptimal - but not as bad as
> when multichannel is negotiated. Interestingly in most cases two network
> connections to the same server (different TCP sockets,but the same mount,
> even in cases where only network adapter) can achieve better performance -
> but still significantly lags Windows (and probably other clients) as in
> Linux we don't keep multiple I/Os in flight at one time (unless different
> files are being read at the same time by different threads).
I think it should be relatively straightforward to make the netfs_readahead()
function generate multiple read requests. If I wasn't handed sufficient pages
by the VM upfront to do two or more read requests, I would need to do extra
expansion. There are a couple of ways this could be done:
(1) I could expand the readahead_control after fully starting a read request
and then create another independent read request, and another for how
ever many we want.
(2) I could expand the readahead_control first to cover however many requests
I'm going to generate, then chop it up into individual read requests.
However, generating larger requests means we're more likely to run into a
problem for the cache: if we can't allocate enough pages to fill out a cache
block, we don't have enough data to write to the cache. Further, if the pages
are just unlocked and abandoned, readpage will be called to read them
individually - which means they likely won't get cached unless the cache
granularity is PAGE_SIZE. But that's probably okay if ENOMEM occurred.
There are some other considerations too:
(*) I would need to query the filesystem to find out if I should create
another request. The fs would have to keep track of how many I/O reqs
are in flight and what the limit is.
(*) How and where should the readahead triggers be emplaced? I'm guessing
that each block would need a trigger and that this should cause more
requests to be generated until we hit the limit.
(*) I would probably need to shuffle the request generation for the second
and subsequent blocks in a single netfs_readahead() call to a worker
thread because it'll probably be in a userspace kernel-side context and
blocking an application from proceeding and consuming the pages already
committed.
David
