On Wed, Feb 22, 2017 at 8:57 PM, Sage Weil <sweil@xxxxxxxxxx> wrote: > On Wed, 22 Feb 2017, John Spray wrote: >> On Wed, Feb 22, 2017 at 11:30 AM, Jeff Layton <jlayton@xxxxxxxxxx> wrote: >> > On Wed, 2017-02-22 at 10:57 +0000, John Spray wrote: >> >> I meant to mention yesterday, I have a branch with the >> >> mds-obeys-blacklist behaviour here: >> >> https://github.com/jcsp/ceph/tree/wip-17980 >> >> >> > >> > Thanks, I'll take a look. >> > >> >> For the other side (blacklisting clients from the MDS), I also >> >> remembered that we currently we have a precedent for special-casing an >> >> mds->OSDMonitor operation, in the form of the MRemoveSnaps message. >> >> >> >> I think the options we currently have for blacklisting clients from the MDS are: >> >> A) Add a new MBlacklist message a la MRemoveSnaps >> >> B) Send a MMonCommand and special case the auth on the mon side to >> >> allow any mds.* entity to run "osd blacklist add" without needing the >> >> usual caps >> >> C) Send a MMonCommand and ask users/scripts setting up Ceph to >> >> explicitly include the mon cap to run the blacklist command. >> >> D) Piggy-back a list of clients to evict on MMDSBeacon, where the MDS >> >> daemon would trim that list once it saw that the blacklist had been >> >> updated to include those clients. >> >> >> >> D is a bit circuitous, but I think it could be interesting as it would >> >> let the mon exercise some discretion on whether to do the client >> >> blacklisting or not, rather than giving the MDSs such power. >> >> >> >> >> > >> > Ugh...I had started working on an approach similar to B/C above, but >> > missed the fact that the mds.* user would need the mon cap. That is a >> > potential problem. >> > >> > A sounds fairly straightforward, and since all of this would require an >> > updated MDS and monitor anyway, it might be the best approach. >> > >> > D sounds clever, but "fiddly". Under what circumstances would you want >> > the monitor to ignore a request from an MDS to blacklist a client? >> >> Not sure, potentially things like implementing a "noevict" flag or >> something that would let admins temporarily put any evictions on hold >> when they know they're about to e.g. bounce a network switch -- >> obviously that would have to mean that MDSs were using the blacklist >> as the only path for eviction, rather than evicting in SessionMap and >> then also blacklisting. It would also be possible to implement that >> by just having the MDS daemons respect a flag in the MDS map though, >> so maybe not the strongest case. >> >> Or maybe we could have some configuration that allowed admins to >> selectively protect certain privileged clients from being >> auto-evicted, that checked targets for eviction/blacklist against that >> map. However, again, if we stored that list in the MDSMap then there >> is nothing preventing implementing that logic MDS side too. >> >> Come to think of it though, I'm being silly, because in case A we >> could also have that message handled by mdsmonitor if we ever wanted >> to have any special logic around handling it, so any special logic >> would really be orthogonal to an A vs. D choice anyway. > > Yeah, I'd go with A. > > And for the 'noevict' case, that logic could also live in the MDS instead > of the mon. So on closer inspection I realised that adding a new message class wasn't as neat as it first seemed -- the MRemoveSnaps is reliable because it gets resent on new osdmaps if it hadn't taken effect, so doesn't need normal resend logic. Also, MRemoveSnaps was already authenticating using an "osd pool rmsnap" MonCap, so it's straightforward to grant the equivalent cap for blacklists, and then send a normal MonCommand to get the resend logic for free. Work in progress is here: https://github.com/jcsp/ceph/commits/wip-17980 John -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
