> On Jul 5, 2024, at 11:58 PM, Mike Snitzer <snitzer@xxxxxxxxxx> wrote: > > On Fri, Jul 05, 2024 at 02:59:31PM +0000, Chuck Lever III wrote: >> >> >>> On Jul 5, 2024, at 10:36 AM, Mike Snitzer <snitzer@xxxxxxxxxx> wrote: >>> >>> On Fri, Jul 05, 2024 at 07:18:29AM -0700, Christoph Hellwig wrote: >>>> On Fri, Jul 05, 2024 at 10:15:46AM -0400, Mike Snitzer wrote: >>>>> NFSv3 is needed because NFSv3 is used to initiate IO to NFSv3 knfsd on >>>>> the same host. >>>> >>>> That doesn't really bring is any further. Why is it required? >>>> >>>> I think we'll just need to stop this discussion until we have reasonable >>>> documentation of the use cases and assumptions, because without that >>>> we'll get hund up in dead loops. >>> >>> It _really_ isn't material to the core capability that localio provides. >>> localio supporting NFSv3 is beneficial for NFSv3 users (NFSv3 in >>> containers). >>> >>> Hammerspace needs localio to work with NFSv3 to assist with its "data >>> movers" that run on the host (using nfs and nfsd). >>> >>> Please just remove yourself from the conversation if you cannot make >>> sense of this. If you'd like to be involved, put the work in to >>> understand the code and be professional. >> >> Sorry, I can't make sense of this either, and I find the >> personal attack here completely inappropriate (and a bit >> hypocritical, to be honest). > > Hi Chuck, > > I'm out-gunned with this good-cop/bad-cop dynamic. I was replying to > Christoph. Who has taken to feign incapable of understanding localio > yet is perfectly OK with flexing like he is an authority on the topic. Well let's try a reality test. Christoph has authored an IETF RFC on pNFS. He's also contributed the pNFS SCSI (and now NVMe) implementation in the Linux server and client. He seems to know the code well enough to offer an informed opinion. > He rallied to your Nacked-By with his chest puffed up and has > proceeded to baselessly shit-talk (did you miss his emails while we > slept last night?). Yes, let's condone and encourage more of that!? > No, I won't abide such toxicity. But thankfully Neil has since called > for him to stop. Alas... > > Earlier today I answered the question about "why NFSv3?" in simple > terms. You and Christoph rejected it. I'm _not_ being evassive. > There isn't a lot to it: "More efficient NFS in containers" _is_ the > answer. > > But hopefully this email settles "why NFSv3?". If not, please help me > (or others) further your understanding by reframing your NFSv3 concern > in terms other than "why NFSv3?". Getting a bit like having to answer > "why is water wet?" > > Why NFSv3? > ---------- > > The localio feature improves IO performance when using NFSv3 and NFSv4 > with containers. Hammerspace has immediate need for the NFSv3 > support, because its "data movers" use NFSv3, but NFSv4 support is > expected to be useful in the future. > > Just because Hammerspace is very invested in pNFS doesn't mean all > aspects are framed in terms of it. > > General statement: > ------------------ > > I wrote maybe ~30% of the entire localio code as it stands at "v11" > and that was focused primarily on adding NFSv4 support and developing > the localio protocol, hooking it into NFS's client initialization and > teardown along with the server (and vice-versa, nfsd lifetime due to > container applications: tearing down nfsd in container while nfs > client actively connected from host). Neil helped refine the localio > protocol part, and he has also looked critically at many aspects and > has a great list of improvements that are needed. Jeff provided > top-notch review of my initial use of SRCU and later the percpu refcnt > for interlocking with the client and server. > > My point: others wrote the majority of localio (years ago). I'm just > trying to shepherd it upstream in an acceptable form. And yes, > localio supporting both NFSv3 and NFSv4 is important to me, > Hammerspace and anyone who'd like more efficient IO with both NFSv3 > and NFSv4 in containers. > > Answering "Why NFSv3?" with questions: > -------------------------------------- > > 1) Why wasn't general NFS localio bypass controversial 3 weeks ago? > Instead (given all inputs, NFSv3 support requirement being one of > them) the use of a "localio protocol" got broad consensus and buyin > from you, Jeff, and Neil. > > I _thought_ we all agreed localio was a worthwhile _auxilliary_ > addition to Linux's NFS client and server (to give them awareness of > each other through nfs_common) regardless of NFS protocol version. > That is why I registered a localio RPC program number with IANA (at > your suggestion, you were cc'd when I applied for it, and you are > named on IANA.org along with Trond and myself for the program number > IANA assigned): > https://www.iana.org/assignments/rpc-program-numbers/rpc-program-numbers.txt > > $ cat rpc-program-numbers.txt | egrep 'Snitzer|Myklebust|Lever' > Linux Kernel Organization 400122 nfslocalio [Mike_Snitzer][Trond_Myklebust][Chuck_Lever] > [Chuck_Lever] Chuck Lever mailto:chuck.lever&oracle.com 2024-06-20 > [Mike_Snitzer] Mike Snitzer mailto:snitzer&kernel.org 2024-06-20 > [Trond_Myklebust] Trond Myklebust mailto:trondmy&hammerspace.com 2024-06-20 > > 2) If we're introducing a general NFS localio bypass feature _and_ > NFSv3 is important to the stakeholder proposing the feature _and_ > NFSv3 support is easily implemented and supported: then why do you > have such concern about localio supporting NFSv3? > > 3) Why do you think NFSv3 unworthy? Is this just a bellweather for > broader opposition to flexfiles (and its encouraging more use of > NFSv3)? Flexfiles is at the heart of NFSv3 use at Hammerspace. I've > come to understand from you and Christoph that the lack of flexfiles > support in NFSD helps fuel dislike for flexfiles. That's a lot for me > to unpack, and pretty far removed from "why NFSv3?", so I'd need more > context than I have if Hammerspace's use of flexfiles is what is > fueling your challenge of localio's NFSv3 support. > > ... > > Reiterating and then expanding on my email above: > > localio supporting NFSv3 is beneficial for NFSv3 users (NFSv3 in > containers). > > Hammerspace needs localio to work with NFSv3 to assist with its > "data movers" that run on the host (using nfs [on host] and nfsd > [within container]). > > Now you can ask why _that_ is.. but it really is pretty disjoint from > the simple matter of ensuring localio support both NFSv3 and NFSv4. > > I've shared that Hammerspace's "data movers" use NFSv3 currently, in > the future they could use NFSv4 as needed. Hence the desire to > support localio with both NFSv3 and NFSv4. [when I picked up the > localio code NFSv4 wasn't even supported yet]. > > I _hope_ I've now answered "why NFSv3?" clearly. > >> I have nothing else to contribute that you won't either >> dismiss or treat as a personal attack, so I can't continue >> this conversation. > > That isn't even a little bit fair... but not taking the bait. > > Neil has been wonderful to work with and I look forward to all future > work with him (localio and beyond). I am not trying to do anything > out of line with this feature. I am and have been actively working > with you, Neil and Jeff for over a month now. I've adapted and > learned, _with_ your and others' help, to the best of my ability. > > I'm trying here, maybe you could say "I'm trying too hard". Well I > just started a new job with Hammerspace after working for Red Hat for > the past 15 years (much of my time spent as the upstream Linux DM > maintainer -- but you know this). I am a capable engineer and I've > proposed the upstreaming of a localio feature that would do well to > land upstream. I've done so in a proficient way all things > considered, always happy to learn new things and improve. I need to > work with you. Hopefully well, and hopefully I can earn your respect, > please just know I'm merely trying to improve NFS. > > Hammerspace would like to get all its Linux kernel NFS innovation > upstream. And I'm trying to do that. localio is my first task and > I've been working on it with focus for the past 2 months since joining > Hammerspace. But you basically know all this, I said all of it to you > at LSF. > > So if you know all these things (I _know_ you do), why are you > treating me in this way? I feel like I'm caught in the middle of some > much bigger divide than anything I've been involved with, caused or > made privy to. Yes, NFS is larger (much larger) and much older than the Linux implementation of it. I'm sorry your new colleagues have not seen fit to help you fit yourself into this picture. > Guess the messenger gets shot sometimes. This is exactly the problem: your attitude of victimhood. You act like our questions are personal attacks on you. Answering "I don't know" or "I need to think about it" or "Let me ask someone" or "Can you explain that further" is perfectly fine. Acknowledging the areas where you need to learn more is a quintessential part of being a professional software engineer. --- I have no strong feelings one way or another about flexfiles. And, I remain a full-throated advocate of the NFSv3 support in the Linux NFS stack. If you get an impression from me, come talk to me first to confirm it. Don't go talk to your colleagues; in particular don't talk to the ones who like to spread a lot of weird ideas about me. You're getting bad information. --- Our question isn't "Why NFSv3?" It's: Can your design document explain in detail how the one existing application (the data mover) will use and benefit from loopback acceleration? It needs to explain why the data mover does not use possibly more suitable solutions like NFSv4.2 COPY. Why are we going to the effort of adding this side car instead of using facilities that are already available? We're not asking for a one sentence email. A one sentence email is not "a response in simple terms". It is a petulant dismissal of our request for more info. We're asking for a real problem statement and use case, in detail, in the design document, not in email. (Go read the requests in this thread again. Honest, that's all we're asking for). --- But finally: We've asked repeatedly for very typical changes and answers, and though sometimes we're met with a positive response, other times we get a defensive response, or "I don't feel like it" or "that's busy work" or "you're wasting my time." That doesn't sound like the spirit of co-operation that I would like to see from a regular contributor, nor do I expect it from someone who is also a Linux kernel maintainer who really ought to know better. So heed Christoph's excellent advice: go eat a Snickers. Calm down. Breathe. None of the rest of us are anywhere near as upset about this as you are right now. -- Chuck Lever
