On Thu, Feb 07, 2019 at 03:54:58PM -0800, Dan Williams wrote: > On Thu, Feb 7, 2019 at 9:17 AM Jason Gunthorpe <jgg@xxxxxxxx> wrote: > > > > Insisting to run RDMA & DAX without ODP and building an elaborate > > revoke mechanism to support non-ODP HW is inherently baroque. > > > > Use the HW that supports ODP. > > > > Since no HW can do disable of a MR, the escalation path is SIGKILL > > which makes it a non-production toy. > > > > What you keep missing is that for people doing this - the RDMA is a > > critical compoment of the system, you can't just say the kernel will > > randomly degrade/kill RDMA processes - that is a 'toy' configuration > > that is not production worthy. > > > > Especially since this revoke idea is basically a DOS engine for the > > RDMA protocol if another process can do actions to trigger revoke. Now > > we have a new class of security problems. (again, screams non > > production toy) > > > > The only production worthy way is to have the FS be a partner in > > making this work without requiring revoke, so the critical RDMA > > traffic can operate safely. > > > > Otherwise we need to stick to ODP. > > Thanks for this it clears a lot of things up for me... > > ...but this statement: > > > The only production worthy way is to have the FS be a partner in > > making this work without requiring revoke, so the critical RDMA > > traffic can operate safely. > > ...belies a path forward. Just swap out "FS be a partner" with "system > administrator be a partner". In other words, If the RDMA stack can't > tolerate an MR being disabled then the administrator needs to actively > disable the paths that would trigger it. Turn off reflink, don't > truncate, avoid any future FS feature that might generate unwanted > lease breaks. We would need to make sure that lease notifications > include the information to identify the lease breaker to debug escapes > that might happen, but it is a solution that can be qualified to not > lease break. In any event, this lets end users pick their filesystem > (modulo RDMA incompatible features), provides an enumeration of lease > break sources in the kernel, and opens up FS-DAX to a wider array of > RDMA adapters. In general this is what Linux has historically done, > give end users technology freedom. To back off the details of this thread a bit... The details of limitations imposed and how they would be tracked within the kernel would be a great thing to discuss face to face. Hence the reason for my proposal as a topic. Ira
