On Fri, 9 Mar 2018, Casey Bodley wrote: > On 03/09/2018 11:55 AM, Sage Weil wrote: > > On Fri, 9 Mar 2018, Casey Bodley wrote: > > > I haven't done much with ceph-ansible, but I'm imagining it looking like > > > this: > > > > > > In the rgw configuration, we'd have a variable like > > > radosgw_management_user > > > that would trigger a 'radosgw-admin user create' command to create it and > > > remember its keys for use during the ceph-mgr deployment. > > > > > > If the ceph-mgr deployment has to happen first, it could always generate > > > its > > > own secret/access keys (which is trivial to do), and supply them later > > > during > > > rgw deployment via 'radosgw-admin user create --access-key=X --secret=Y'. > > I think this is missing the bigger picture. Setting aside the key issue > > for a minute, there needs to be some API endpoint that allows you to > > manipulate the zones/zone groups/realms (e.g., to create the radosgw > > cluster to begin with). Creating an initial key for that zone is just one > > piece of that. > > > > For example, a dashboard user should be able to click on the RGW tab and > > create a new realm or zone and then kick off work to instantiate the > > radosgw daemons to serve it (via kubernetes, ansible, or whatever). > > Hi Sage, > > I didn't know that we were looking to drive new cluster deployments through > ceph-mgr. But I think that the multisite configuration steps to make that work > belong in the deployment tool itself. Ali has done work on this for > ceph-ansible at https://github.com/ceph/ceph-ansible/pull/1944, which runs all > of the radosgw-admin commands on the new cluster to add it to an existing > multisite realm. I think the general goal is that any management functions beyond the initial bootstrap of the cluster (mon + mgr) can be driven via the management ui. But even setting aside the multi-cluster parts, the very first thing I would expect to see on the RGW pane of the dashboard is a via of the realms, zonegroups, and zones, with a bunch of 'create' buttons. And the first thing you'd see on a fresh cluster is no zones--I don't think we want to force the user to predeclare that they will be creating a new realm/zg/zone when they create the cluster. Even for the multi-cluster parts, I would expect to see a limited view of that state. The RBD panel, for instance, shows all the rbd-mirror daemons and their state. Operations like switching masters or triggering a period change or whatever seem like a natural fit here. Even if we decide these operations don't fit or don't belong in the per-cluster dashboard, we'll want them in some meta-dashbaord (e.g., cloudforms), and we'll want an API that that system can trigger to make things happen. That could be ansible, yes, but it seems like we'd want to keep our options open. As I understand it the current tie-in with radosgw-admin CLI is historical. If we can make just the realm/zg/zone management functions usable via a librgw-something (vai the mgr or radosgw-admin cli), wouldn't that be more flexible? > > Once we have a radosgw running and a key installed, then we can talk > > to the radosgw admin API. > > > > Relying on ansible to run CLI commands feels very backward here. If the > > radosgw-admin CLI is the only tool that can manipulate the realm and > > zone maps then it seems like that is the piece to fix? > > > > Am I missing something? > > sage > > > > The CLI commands were for bootstrapping on cluster creation, isn't that a > normal use of ansible? > > As I see it, the issue is that radosgw-admin, the radosgw admin apis, and any > library interfaces we could invent still only have a view of the local ceph > cluster. I'm happy to help build admin apis for the local zone configuration > parts that are missing. But outside of the 'radosgw-admin period commit' > command that pushes local changes to the master zone, any coordination between > clusters would have to come in at a higher level. There will inevitably be some bootstrap stage to create new clusters and some step to teach them about each other, but once that happens the clusters *do* talk to each other. Given that we do have important multi-cluster replication fucntions in both RBD and RGW and there is no "higher level" to delegate to (besides a CLI) I'm worried about painting ourselves into a corner we'll regret later... I suspect there are some headaches with regard to the current structure of radosgw-admin code that make wrapping it up in a library with python bindings a painful exercise, but that doesn't mean it isn't worth the effort for just the realm management pieces? sage -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
