On Tue, Sep 29, 2020 at 3:33 PM Travis Nielsen <tnielsen@xxxxxxxxxx> wrote: > > Sebastian and fellow orchestrators, > > Some questions have come up recently about issues in the Rook > orchestrator module and its state of disrepair. Patrick, Varsha, and I > have been discussing these recently as Varsha has been working on the > module. Before we fix all the issues that are being found, I want to > start a higher level conversation. I’ll join the leads meeting > tomorrow to discuss, and would be good to include in the Monday > orchestrator agenda as well, which unfortunately I haven’t been able > to attend recently... > > First, Rook is driven by the K8s APIs, including CRDs, an operator, > the CSI driver, etc. When the admin needs to configure the Ceph > cluster, they create the CRDs and other resources directly with the > K8s tools such as kubectl. Rook does everything with K8s patterns so > that the admin doesn’t need to leave their standard administration > sandbox in order to configure Rook or Ceph. If any Ceph-specific > command needs to be run, the rook toolbox can be used. However, we > prefer to avoid the toolbox for common scenarios that should have CRDs > for declaring desired state. > > The fundamental question then is, **what scenarios require the Rook > orchestrator mgr module**? The module is not enabled by default in > Rook clusters and I am not aware of upstream users consuming it. > > The purpose of the orchestrator module was originally to provide a > common entry point either for Ceph CLI tools or the dashboard. This > would provide the constant interface to work with both Rook or cephadm > clusters. Patrick pointed out that the dashboard isn’t really a > scenario anymore for the orchestrator module. Is that true? [1] > If so, the only > remaining usage is for CLI tools. And if we only have the CLI > scenario, this means that the CLI commands would be run from the > toolbox. But we are trying to avoid the toolbox. We should be putting > our effort into the CRDs, CSI driver, etc. > > If the orchestrator module is creating CRs, we are likely doing > something wrong. We expect the cluster admin to create CRs. > > Thus, I’d like to understand the scenarios where the rook orchestrator > module is needed. If there isn’t a need anymore since dashboard > requirements have changed, I’d propose the module can be removed. I don't have a current stake in the outcome, but I could foresee the future need/desire for letting the Ceph cluster itself spin up resources on-demand in k8s via Rook. Let's say that I want to convert an XFS on RBD image to CephFS, the MGR could instruct the orchestrator to kick off a job to translate between the two formats. I'd imagine the same could be argued for on-demand NFS/SMB gateways or anywhere else there is a delta between a storage administrator setting up the basic Ceph system and Ceph attempting to self-regulate/optimize. > Thanks, > Travis > Rook > [1] https://tracker.ceph.com/issues/46756 -- Jason
