On Tue, 12 Nov 2019, Sebastian Wagner wrote: > > Am 12.11.19 um 15:52 schrieb Sage Weil: > > Right now the way ceph-daemon is used by the ssh orchestrator is designed > > to minimize the dependencies/setup complexity. The only requirements for > > a host to be added to the cluster are > > > > - python (2 or 3) > > - systemd > > - either podman or docker installed > > - the ceph cluster's pub key in /root/.ssh/authorized_keys > > > > No other software (including Ceph) needs to be installed. The mgr/ssh > > module invokes ceph-daemon on the remote host by running /usr/bin/python > > over ssh and piping the cluster's version of ceph-daemon to stdin. > > > > The downside to this approach is that some users might not like the idea > > of ceph having an ssh key with root access. For large clusters I'm not > > sure how much this really matters--if you pwn ceph you can delete TB to PB > > of data so do you really care if someone has root?--but for hyperconverged > > cases this might be a problem. > > having the possibility to call sudo ceph-daemon sounds like a good thing > to me! > > > > > One alternative might be to > > > > - create a ceph user on the node, and put the cluster's key in > > that user's authorized_keys > > - install a package that includes ceph-daemon (/usr/bin/ceph-damaen) > > - install an /etc/sudoers.d/ceph file that lets the ceph user > > 'sudo ceph-daemon ...' > > > > Cons: > > - This makes the bootstrap process slightly more complicated: (1) install > > package, (2) create user, (3) install ssh key (vs just #3). > > - The remote version of ceph-daemon can get out of sync with the > > cluster.. either stale and missing some feature, or even too new and > > not behaving the way the cluster expects.> > > Pros: > > - This limits the attack surface area (if someone manages to get the > > cluster's ssh key) to the functions that ceph-daemon implements, vs > > full root. > > > > We could mitigate the 'keep ceph-daemon up to date' problem somewhat by > > implementing a 'ceph-daemon update' function that will apt/dnf/yum install > > ceph-daemon on the local host, so that the cluster could self-update the > > remote host. > > I guess there are better tools for that job for installing and updateing > software (Ansible, Salt, pip). Yeah... I don't like the idea of having to think about packaging tools, repo locations, and the like. > > Or... we could skip the package entirely and install the ceph-daemon > > script in /home/ceph/ceph-daemon, and include an update function that > > updates the script in place. That is less complicated than knowing how to > > apt/dnf/yum install a package for all the random distros and repo location > > combinations (one of the biggest benefits of containers IMO). But it > > still requires some sort of process to keep ceph-daemon up to date. > > Maybe if we always pass an md5sum to ceph-daemon whenever we invoke it to > > assert that we are running the version we want, and if there is a > > mismatch, ceph-daemon bails out with a special exit code that triggers and > > update and retry? > > Sounds like re-implementing pip from scratch or so. It sounds a bit like pip, but I don't think pip would actually work here. The idea is to keep ceph-daemon in sync with the remote mgr/ssh module, not to have the latest/greatest, or whatever broken thing pypi might have. Nor do we want to add a new external web service dependency to the process--right now we only depend on a container registry, and that may be dockerhub by default, but it could be anything you specify in your --image argument. > I'm +1 for adding some validation to make sure we're running with a > compatible version of ceph-daemon. Maybe defining an API version of > ceph-daemon. > > But I guess we need to properly install ceph-daemon via RPM for > downstream (We need to install ceph-daemon for the bootstrap anyway). > For the community, I'd also be ok with something like pip install > --update or password-less ssh rook access. I have a feeling distros/products will have a packaged ceph-daemon available for bootstrapping, and perhaps 'ceph-daemon shell'. But IMO using a packaged version for mgr/ssh is a fragile/broken approach because the ceph cluster will be upgrading itself, and part of that upgrade will invitably involve updated versions of ceph-daemon. And we don't want ceph to have to learn about package managers. Again, the only reason we'd write ceph-daemon to disk on the remote node *at all* is so that we can sudo run it. ... The problem with this, now that I think about it, is that $cephuser can sudo /home/$cephuser/ceph-daemon and also rewrite it, which makes it useless as a security barrier. I think ceph-daemon has to be root-owned and in a root-owned directory (/root/ceph-daemon?) in order for sudo to be potentially useful here. But then in order for the script to be updateable, mgr/ssh sshing in as $cephuser needs to be able to update it. Which means that if someone has that ssh key they can always update ceph-daemon to run /bin/bash (or whatever) and circumvent any protection. So I think ability to have mgr/ssh keep ceph-daemon in sync with itself is fundamentally incompatible with providing any meaningful protection. Sigh... sage _______________________________________________ Dev mailing list -- dev@xxxxxxx To unsubscribe send an email to dev-leave@xxxxxxx
