On Thu, May 09, 2019 at 12:02:17AM -0500, Vijay Chidambaram wrote: > As we have stated on multiple times on this and other threads, the > intention is *not* to come up with one set of crash-recovery > guarantees that every Linux file system must abide by forever. Ted, > you keep repeating this, though we have never said this was our > intention. > > The intention behind this effort is to simply document the > crash-recovery guarantees provided today by different Linux file > systems. Ted, you question why this is required at all, and why we > simply can't use POSIX and man pages. But who is this documentation targeted towards? Who is it intended to benefit? Most application authors do not write applications with specific file systems in mind. And even if they do, they can't control how their users are going to use it. > FWIW, I think the position of "if we don't write it down, application > developers can't depend on it" is wrong. Even with nothing written > down, developers noticed they could skip fsync() in ext3 when > atomically updating files with rename(). This lead to the whole ext4 > rename-and-delayed-allocation problem. The much better path, IMO, is > to document the current set of guarantees given by different file > systems, and talk about what is intended and what is not. This would > give application developers much better guidance in writing > applications. If we were to provide that nuance, that would be much better, I would agree. It's not what the current crash consistency guarantees provides, alas. I'd also want to talk about what is guaranteed *first*; documenting the current state of affairs, some of which may be subject to change and the result of the implementation, is far less important. So I'd prefer that "documentation of current behavior" be the last thing in the document --- perhaps in an appendix --- and not the headliner. Indeed, I'd use the ext3 O_PONIES discussion as a prime example of the risk if we were to just "document current practice" and stop there. It's the fact that your crash consistency guarantees draft, claims to "document current practice", and at the same time, uses the word "guarantee" which causes red flags to go up for me. If we could separate those two, that would be very helpful. And if the current POSIX guarantees are too vague, my preference would be to first determine what application authors would find more useful in terms stricter guarantees, and provide those guarantees as we find them. We can always add more guarantees later. Taking guarantees away is much harder. And guarantees by defintion always restrict freedom of action, so this is an engineering tradeoff. Let's provide those guarantees when it actually improves application performance, and not Just Because. It might also be that defining new system calls, like fbarrier() and fdatabarrier() is a better approach rather than retconning new semantics on top of fsync(). I just think a principled design approach is better rather than taking existing semantics and slapping the word "guarantee" in the title of said documentation. I will also say that I have no problems with documenting strong metadata ordering if it has nothing to do with fsync(). That makes sense. The moment that you try to also bring data integrity into the mix, and give examples of what happens if you call fsync(), that it goes beyond strong metadata ordering. So if you want to document what happens without fsync, ext4 can probably get on board with them. Unfortuantely, in addition to including the word "guarantee", the current crash consistency draft also includes the word "fsync". > 4. Apart from developers, a document like this would also help > academic researchers understand the current state-of-the-art in > crash-recovery guarantees and the different choices made by different > file systems. It is non-trivial to understand this without > documentation. It's also very hard to undertand this without taking performance constraints and implementation choices into account. It's trivially easy to give super-strong crash-recovery guarantees. But if it sacrifices performance, is it really "state-of-the-art"? Worse, different applications may want different guarantees, and may want different crash consistency vs. performance tradeoffs. This is why in general, the concept of providing new interfaces where the application can state more explicitly what they want is much more appealing to me. When I have discussions with Amir, he doesn't just want strong guarantees; he wants specific guarantees with zero overhead, and our discussions have been in how to we manage that tension between those two goals. And it's much easier to achieve this in terms of very specific cases, such as what happens when you link an O_TMPFILE file into a directory. Cheers, - Ted
