On Tue, Sep 01, 2020 at 11:14:53AM -0400, Theodore Y. Ts'o wrote: > On Tue, Sep 01, 2020 at 10:52:05AM -0400, Theodore Y. Ts'o wrote: > > On Tue, Sep 01, 2020 at 01:34:05PM +1000, Dave Chinner wrote: > > > > > > But, unlike your implication that this is -really complex and hard > > > to do-, it's actually relatively trivial to do with the XFS > > > implementation I mentioned as each ADS stream is a fully fledged > > > inode that can point to shared data extents. If you can do data > > > manipulation on a regular inode, you'll be able to do it on an ADS, > > > and that includes copying ADS streams via reflink. > > > > Is the reflink system call on a file with ADS's atomic, or not? We can implement as wholly atomic if we want to, yes. > > What > > if there are a million files is ADS hierarchy which is 100 > > subdirectories deep in some places, comprising several TB's worth of > > data? Is that all going to fit in a single XFS transaction? No, but we solved that "unbound transaction size" problem years ago for reflink and reverse mapping updates. We have constructs like intents and rolling atomic transactions to allow largely unbound modifications to run without being limited by the maximum size of a single transaction or even the size of the journal... > > What if > > you crash in the middle of it? Intents track modification progress and allow recovery to restart from exactly where the journal says the operation had got to.... > > Is a partially reflinked copy of an > > ADS file OK? Yes. > > Or a reflinked ADS file missing some portion of the > > alternate data streams? I don't know what this is refering to. > Oh, and if the answer is that the ADS inodes should be reflinked > individually in userspace, wonderful! You can do that too, if you want - FI_CLONE/clone_file_range work on open file descriptors, not file names, so if you can represent an ADS as a file descriptor, you can clone it, dedupe it, etc, just like you can with an other user data file. > An ADS inode could then just be > a directory, like it was in the NeXT operating system, and copying an > ADS file could *also* be done in userspace, as a cp -r. :-) And then you lose all the atomicity and recoverability of reflink copies. So, not they are not equivalent, and demonstrate where the limits of representing ADS as "directories in a file" reduce the ability to atomically manipulate ADS as group of objects attached to a file... Cheers, Dave. -- Dave Chinner david@xxxxxxxxxxxxx
