On Wed, May 22, 2019 at 7:25 PM Yurii Zubrytskyi <zyy@xxxxxxxxxx> wrote: > > > Hang on, fuse does use caches in the kernel (page cache, > > dcache/icache). The issue is probably not lack of cache, it's how the > > caches are primed and used. Did you disable these caches? Did you > > not disable invalidation for data, metadata and dcache? In recent > > kernels we added caching readdir as well. The only objects not cached > > are (non-acl) xattrs. Do you have those? > Android (which is our primary use case) is constantly under memory > pressure, so caches > don't actually last long. Our experience with FOPEN_KEEP_CACHE has > shown that pages are > evicted more often than the files are getting reopened, so it doesn't > help. FUSE has to re-read > the data from the backing store all the time. What would benefit many fuse applications is to let the kernel transfer data to/from a given location (i.e. offset within a file). So instead of transferring data directly in the READ/WRITE messages, there would be a MAP message that would return information about where the data resides (list of extents+extra parameters for compression/encryption). The returned information could be generic enough for your needs, I think. The fuse kernel module would cache this mapping, and could keep the mapping around for possibly much longer than the data itself, since it would require orders of magnitude less memory. This would not only be saving memory copies, but also the number of round trips to userspace. There's also work currently ongoing in optimizing the overhead of userspace roundtrip. The most promising thing appears to be matching up the CPU for the userspace server with that of the task doing the request. This can apparently result in 60-500% speed improvement. > We didn't use xattrs for the FUSE-based implementation, but ended up > requiring a similar thing in > the Incremental FS, so the final design would have to include them. > > > Re prefetching data: > > there's the NOTIFY_STORE message. > To add to the previous point, we do not have the data for prefetching, > as we're loading it page-by-page > from the host. We had to disable readahead for FUSE completely, > otherwise even USB3 isn't fast enough Understood. Did you re-enable readahead for the case when the file has been fully downloaded? Thanks, Miklos
