Thanks a lot, Edward, for sharing this thorough description! I filed a tracker to record your findings and improve this set-up process ( https://tracker.ceph.com/issues/53127). Additionally, did you try with "cephadm shell -n mgr.<hostname> -m <mountpoint>"? If I'm not wrong, that should give you a shell where filesystems are exactly the same as in the target ceph-mgr instance (and if you launch this co-located with the target ceph-mgr daemon, I guess it might physically share some mounts... @Sebastian Wagner <sewagner@xxxxxxxxxx> ?). Kind Regards, Ernesto On Fri, Oct 29, 2021 at 5:49 PM Edward R Huyer <erhvks@xxxxxxx> wrote: > Ok, I figured out how to feed the containerized SAML module the x509 > certificate. The solution is dumb and hacky, but it appears to work. I’m > putting what I did and why it worked here in case others come looking. > > > > The cert and key need to be put somewhere on the host filesystem that is > also mounted inside the container, so the daemon running inside the > container can see it. The paths you give the “ceph dashboard sso setup > saml2” command have to match the paths inside the container. BUT that > command also checks the paths against the *host’s* filesystem, so the path > *also* has to match there! And lastly, I think the permissions on the > files need to be such that the daemons running as the ceph user can read > them, but I’m not positive about that. > > > > So here’s what you do. > > > > Generate your cert and key: > > openssl genrsa -out private.key 1024 > > openssl req -new -x509 -key private.key -out publickey.cer -days 365 > > > > Copy your cert and key to the mgr data directory on the host, and set > their ownership/permissions: > > cp publickey.cer /var/lib/ceph/{UUID}/mgr.{hostname}/ > > cp private.key /var/lib/ceph/{UUID}/mgr.{hostname}/ > > chown ceph:ceph /var/lib/ceph/{UUID}/mgr.{hostname}/publickey.cer > > chown ceph:ceph /var/lib/ceph/{UUID}/mgr.{hostname}/private.key > > chmod 600 /var/lib/ceph/{UUID}/mgr.{hostname}/publickey.cer > > chmod 600 /var/lib/ceph/{UUID}/mgr.{hostname}/private.key > > > > Inside the mgr container, they now show up in the path > “/var/lib/ceph/mgr/{clustername}-{hostname}” (clustername is probably > “ceph”). > > Note that this is different from the path on the host! > > > > Since the paths don’t match, we have to create a symbolic link on the host > so the files exist in the same path on both the host filesystem and > container: > > ln -s /var/lib/ceph/{UUID}/mgr.{hostname}/ > /var/lib/ceph/mgr/{clustername}-{hostname} > > > > Now we can finally run the setup: > > ceph dashboard sso setup saml2 <ceph_dashboard_base_url> <idp_metadata> > <idp_username_attribute> <idp_entity_id> > /var/lib/ceph/{UUID}/mgr.{hostname}/publickey.cer > /var/lib/ceph/{UUID}/mgr.{hostname}/private.key > > > > Note that idb_username_attribute and idp_entity_id are NOT optional when > specifying a certificate and key. > > > > Lastly, just to clean up, remove the (I assume now unnecessary, but I > haven’t tested much) symbolic link. > > rm /var/lib/ceph/mgr/{clustername}-{hostname} > > > > And that’s it. Hopefully this will become more straightforward in the > future, but in the meantime, maybe it will be helpful to someone else. > > > > *From:* Edward R Huyer > *Sent:* Wednesday, October 27, 2021 9:31 AM > *To:* 'Ernesto Puerta' <epuertat@xxxxxxxxxx> > *Cc:* Yury Kirsanov <y.kirsanov@xxxxxxxxx>; ceph-users@xxxxxxx > *Subject:* RE: Re: Doing SAML2 Auth With Containerized mgrs > > > > Thank you for the reply. Even if there’s a good reason for the CLI tool > to not send the contents of the files, I feel like the docs should at least > have “this is how we recommend you handle this if you’re using a > containerized (e.g. cephadm) deployment”. > > > > Speaking of which, do you have any specific suggestions as to how to > approach this? I’m not familiar enough the details of the cephadm-deploy > containers specifically or containers in general to know where to start. > > > > *From:* Ernesto Puerta <epuertat@xxxxxxxxxx> > *Sent:* Wednesday, October 27, 2021 6:53 AM > *To:* Edward R Huyer <erhvks@xxxxxxx> > *Cc:* Yury Kirsanov <y.kirsanov@xxxxxxxxx>; ceph-users@xxxxxxx > *Subject:* Re: Re: Doing SAML2 Auth With Containerized mgrs > > > > Hi Edward, > > > > You don't need to worry about keeping those certs persistently, the Ceph > Dashboard does that for you (they're persisted inside the ceph-mon KV > store). You just need to ensure that the paths you provide are reachable to > the ceph-mgr daemon. And I agree: that's a bit tricky with containerized > deployments (it was originally designed for bare-metal ones in mind). We > could improve that by making the Ceph CLI tool read CephFile-type args and > send the contents of these files, instead of sending the paths to the > daemon (I don't know what was the original intent of this behaviour, maybe > avoid sending sensitive data unencrypted or avoiding the multiple loggers > that might dump what's sent from the Ceph CLI). > > > > Kind Regards, > > Ernesto > > > > > > On Mon, Oct 25, 2021 at 6:21 PM Edward R Huyer <erhvks@xxxxxxx> wrote: > > No worries. It's a pretty specific problem, and the documentation could > be better. > > -----Original Message----- > From: Yury Kirsanov <y.kirsanov@xxxxxxxxx> > Sent: Monday, October 25, 2021 12:17 PM > To: Edward R Huyer <erhvks@xxxxxxx> > Cc: ceph-users@xxxxxxx > Subject: Re: Doing SAML2 Auth With Containerized mgrs > > Hi Edward, > Yes, you probably are right, I thought about dashboard SSL certificate, > not the SAML2, sorry for that. > > Regards, > Yury. > > On Tue, Oct 26, 2021 at 3:10 AM Edward R Huyer <erhvks@xxxxxxx> wrote: > > > I don’t think that’s correct? I already have a certificate set up for > > HTTPS, and it doesn’t show up in the SAML2 configuration. Maybe I’m > > mistaken, but I think the SAML2 cert is separate from the regular > > HTTPS cert? > > > > > > > > *From:* Yury Kirsanov <y.kirsanov@xxxxxxxxx> > > *Sent:* Monday, October 25, 2021 11:52 AM > > *To:* Edward R Huyer <erhvks@xxxxxxx> > > *Cc:* ceph-users@xxxxxxx > > *Subject:* Re: Doing SAML2 Auth With Containerized mgrs > > > > > > > > *CAUTION: This message came from outside RIT. If you are unsure about > > the source or content of this message, please contact the RIT Service > > Center at > > 585-475-5000 or help.rit.edu <http://help.rit.edu> before clicking > > links, opening attachments or responding.* > > > > Hi Edward, > > > > You need to set configuration like this, assuming that certificate and > > key are on your local disk: > > > > ceph mgr module disable dashboard > > ceph dashboard set-ssl-certificate -i <your_certificate>.crt ceph > > dashboard set-ssl-certificate-key -i <your_certificate_key>.key ceph > > config-key set mgr/cephadm/grafana_crt -i <your_certificate>.crt ceph > > config-key set mgr/cephadm/grafana_key -i <your_certificate_key>.key > > ceph orch reconfig grafana ceph mgr module enable dashboard > > > > Hope this helps! > > > > Regards, > > Yury. > > > > > > > > On Tue, Oct 26, 2021 at 2:45 AM Edward R Huyer <erhvks@xxxxxxx> wrote: > > > > Continuing my containerized Ceph adventures.... > > > > I'm trying to set up SAML2 auth for the dashboard (specifically > > pointing at the institute Shibboleth service). The service requires > > the use of the > > x509 certificates. Following the instructions in the documentation ( > > https://docs.ceph.com/en/latest/mgr/dashboard/#dashboard-sso-support ) > > leads to an error about the certificate file not existing. > > > > Some digging suggests that's because the daemon is looking in the > > container's filesystem rather than the physical host's filesystem. > > That makes some sense, but it annoying. > > > > So my question is: How do I get the cert and key file into the > > container's filesystem in a persistent way? cephadm enter --name > > "mgr.hostname" results in a "no such container" error. cephadm shell > > --name "mgr.hostname" works, but changes don't persist. > > > > Any suggestions about this problem specifically, authing the dashboard > > against Shibboleth, or SAML2 in general? > > > > ----- > > Edward Huyer > > Golisano College of Computing and Information Sciences Rochester > > Institute of Technology Golisano 70-2373 > > 152 Lomb Memorial Drive > > Rochester, NY 14623 > > 585-475-6651 > > erhvks@xxxxxxx<mailto:erhvks@xxxxxxx> > > > > Obligatory Legalese: > > The information transmitted, including attachments, is intended only > > for the person(s) or entity to which it is addressed and may contain > > confidential and/or privileged material. Any review, retransmission, > > dissemination or other use of, or taking of any action in reliance > > upon this information by persons or entities other than the intended > > recipient is prohibited. If you received this in error, please contact > > the sender and destroy any copies of this information. > > > > _______________________________________________ > > ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an > > email to ceph-users-leave@xxxxxxx > > > > > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an > email to ceph-users-leave@xxxxxxx > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |