On 7/7/21 1:59 PM, Omar Sandoval wrote:
On Fri, Jun 25, 2021 at 02:07:59PM -0700, Omar Sandoval wrote:
On Fri, Jun 25, 2021 at 09:16:15AM -0700, Linus Torvalds wrote:
On Thu, Jun 24, 2021 at 8:38 PM Matthew Wilcox <willy@xxxxxxxxxxxxx> wrote:
Does it make any kind of sense to talk about doing this for buffered I/O,
given that we can't generate them for (eg) mmaped files?
Sure we can.
Or rather, some people might very well like to do it even for mutable
data. In fact, _especially_ for mutable data.
You might want to do things like "write out the state I verified just
a moment ago", and if it has changed since then, you *want* the result
to be invalid because the checksums no longer match - in case somebody
else changed the data you used for the state calculation and
verification in the meantime. It's very much why you'd want a separate
checksum in the first place.
Yeah, yeah, you can - and people do - just do things like this with a
separate checksum. But if you know that the filesystem has internal
checksumming support _anyway_, you might want to use it, and basically
say "use this checksum, if the data doesn't match when I read it back
I want to get an IO error".
(The "data doesn't match" _could_ be just due to DRAM corruption etc,
of course. Some people care about things like that. You want
"verified" filesystem contents - it might not be about security, it
might simply be about "I have validated this data and if it's not the
same data any more it's useless and I need to re-generate it").
Am I a big believer in this model? No. Portability concerns (across
OS'es, across filesystems, even just across backups on the same exact
system) means that even if we did this, very few people would use it.
People who want this end up using an external checksum instead and do
it outside of and separately from the actual IO, because then they can
do it on existing systems.
So my argument is not "we want this". My argument is purely that some
buffered filesystem IO case isn't actually any different from the
traditional "I want access to the low-level sector hardware checksum
data". The use cases are basically exactly the same.
Of course, basically nobody does that hw sector checksum either, for
all the same reasons, even if it's been around for decades.
So my "checksum metadata interface" is not something I'm a big
believer in, but I really don't think it's really all _that_ different
from the whole "compressed format interface" that this whole patch
series is about. They are pretty much the same thing in many ways.
I see the similarity in the sense that we basically want to pass some
extra metadata down with the read or write. So then do we want to add
preadv3/pwritev3 for encoded I/O now so that checksums can use it in the
future? The encoding metadata could go in this "struct io_how", either
directly or in a separate structure with a pointer in "struct io_how".
It could get messy with compat syscalls.
Ping. What's the path forward here? At this point, it seems like an
ioctl is the path of least resistance.
At this point we've been deadlocked on this for too long. Put it in a btrfs
IOCTL, if somebody wants to extend it generically in the future then godspeed,
we can tie into that interface after the fact. Thanks,
Josef
