Hi Linus,
Adding Jens, because he's one of the main splice people. You do seem
to be stepping on his work ;)
Jens, see
https://lore.kernel.org/lkml/0cfd9f02-dea7-90e2-e932-c8129b6013c7@xxxxxxxxx
Ok, thanks! Maybe Jens should apear in the output of:
scripts/get_maintainer.pl fs/splice.c
On Thu, Feb 9, 2023 at 5:56 AM Stefan Metzmacher <metze@xxxxxxxxx> wrote:
So we have two cases:
1. network -> socket -> splice -> pipe -> splice -> file -> storage
2. storage -> file -> splice -> pipe -> splice -> socket -> network
With 1. I guess everything can work reliable [..]
But with 2. there's a problem, as the pages from the file,
which are spliced into the pipe are still shared without
copy on write with the file(system).
Well, honestly, that's really the whole point of splice. It was
designed to be a way to share the storage data without having to go
through a copy.
I'm wondering if there's a possible way out of this, maybe triggered by a new
flag passed to splice.
Not really.
So basically, you cannot do "copy on write" on a page cache page,
because that breaks sharing.
You *want* the sharing to break, but that's because you're violating
what splice() was for, but think about all the cases where somebody is
just using mmap() and expects to see the file changes.
You also aren't thinking of the case where the page is already mapped
writably, and user processes may be changing the data at any time.
I do because we're using that in our tdb library, but I hoped there would be
a way out...
I looked through the code and noticed the existence of IOMAP_F_SHARED.
Yeah, no. That's a hacky filesystem thing. It's not even a flag in
anything core like 'struct page', it's just entirely internal to the
filesystem itself.
Ok, I guess it's used for shared blocks in the filesystems,
in order to support things like cow support in order to allow
snapshots, correct?
Is there any other way we could archive something like this?
I suspect you simply want to copy it at splice time, rather than push
the page itself into the pipe as we do in copy_page_to_iter_pipe().
Because the whole point of zero-copy really is that zero copy. And the
whole point of splice() was to *not* complicate the rest of the system
over-much, while allowing special cases.
Linux is not the heap of bad ideas that is Hurd that does various
versioning etc, and that made copy-on-write a first-class citizen
because it uses the concept of "immutable mapped data" for reads and
writes.
Ok, thanks very much for the detailed feedback!
Now, I do see a couple of possible alternatives to "just create a stable copy".
For example, we very much have the notion of "confirm buffer data
before copying". It's used for things like "I started the IO on the
page, but the IO failed with an error, so even though I gave you a
splice buffer, it turns out you can't use it".
And I do wonder if we could introduce a notion of "optimistic splice",
where the splice works exactly the way it does now (you get a page
reference), but the "confirm" phase could check whether something has
changed in that mapping (using the file versioning or whatever - I'm
hand-waving) and simply fail the confirm.
That would mean that the "splice to socket" part would fail in your
chain, and you'd have to re-try it. But then the onus would be on
*you* as a splicer, not on the rest of the system to fix up your
special case.
That idea sounds fairly far out there, and complicated and maybe not
usable. So I'm just throwing it out as a "let's try to think of
alternative solutions".
That sounds complicated and still racy.
Any comment about the idea of having a preadv2() flag that
asks for a dma copy with something like async_memcpy() instead
of the default that ends up in copy_user_enhanced_fast_string()?
If that would be possible, a similar flag would also be possible
for splice() in order to dma copy the pages into the pipe.
metze
