When certificate auto-generation is configured (by setting generate_cert to true in the spec) cephadm will take care of generating the certificates for each host. This feature was introduced recently so I'm not sure if it's well documented or not (I'll check that later). We are introducing this functionality little by little for all the services and cephadm is becoming the root CA for the self-signed certificates needed by the different ceph services (when the user doesn't provide his own). Then you can add cephadm root CA as a trusted certificate in your chain. Anyway, it seems like you have a good enough work around for your use case. Let's continue the discussion on the issue tracker. Best Regards, Redo. On Fri, Jan 17, 2025 at 1:17 PM Alex Hussein-Kershaw (HE/HIM) < alexhus@xxxxxxxxxxxxx> wrote: > Thanks for the suggestion. It may be that it does fit my use case but I > wasn't sure from the docs. > > I have a self-signed CA that I use for my deployment, can pass a root CA > cert and key in to Cephadm to have it auto-generate a server certificate as > a child of that? > > I noticed in the docs there is no mention of those and I had a very brief > look at the code and didn't see the ability to do that. It looked like > perhaps Cephadm would generate its own root cert and sign with that? > > I've got a bit of baggage around this area (clients, cert rotation > processes) that probably complicate using this also. > > I'm probably going to shy away from this for now. I still think a sensible > fix here is to not add the defaulted ssl_certificate option if it's passed > as an extra_frontend_arg, which allows my use case without a mainline > change to behaviour. > > Thanks, > Alex > ------------------------------ > *From:* Redouane Kachach > *Sent:* Friday, January 17, 2025 11:14 AM > *To:* Alex Hussein-Kershaw (HE/HIM) > *Cc:* ceph-users > *Subject:* Re: [EXTERNAL] Re: Cephadm: Specifying RGW Certs > & Keys By Filepath > > Thank you Alex, > > Reading the description of the tracker > https://tracker.ceph.com/issues/69567 sounds like the outcome is very > similar to when using the self-signed auto-generated certificates by > cephadm. Wouldn't that be a viable option? I mean is there a big difference > between self-signing the certificates by yourself and having cephadm doing > that for you (unless your Org has strict rules around that!)? > > > > On Fri, Jan 17, 2025 at 10:19 AM Redouane Kachach <rkachach@xxxxxxxxxx> > wrote: > > I see, unfortunately I can't see an easy way to avoid that. With the > current code you will get either port= or ssl_port= depending on whether > you enable ssl or not. At the end of the day the result get written in the > configuration variable rgw_frontends (ceph config ls | grep > rgw_frontends). > > I'll recommend opening a ticket with a feature request explaining your use > case and why the current behavior doesn't meet your needs. > > Best, > Redo. > > > > > On Fri, Jan 17, 2025 at 9:51 AM Alex Hussein-Kershaw (HE/HIM) < > alexhus@xxxxxxxxxxxxx> wrote: > > Moring Redo, > > I had dropped the "rgw_frontend_port" as I was providing it via > "ssl_port". However I tested with it back in also: > > service_type: rgw > service_id: rgw > service_name: rgw.rgw > placement: > count_per_host: 1 > host_pattern: '*' > extra_container_args: > - -v > - /etc/pki:/etc/pki:ro > - -v > - /etc/ssl/:/etc/ssl:ro > spec: > rgw_frontend_port: 7480 > rgw_frontend_extra_args: > - ssl_port=7480 > - ssl_certificate=/etc/ssl/certs/server.crt > - ssl_private_key=/etc/ssl/private/server.key > rgw_realm: geored_realm > rgw_zone: siteA > rgw_zonegroup: geored_zg > > Results in config: > > beast port=7480 ssl_port=7480 ssl_certificate=/etc/ssl/certs/server.crt > ssl_private_key=/etc/ssl/private/server.key > > > ...and I then get errors in logs because it's trying to bind to the same > port twice. > > failed to bind address 0.0.0.0:7480: Address already in use > > Dropping the "ssl_port" just brings it up as a HTTP endpoint. > > beast port=7480 ssl_certificate=/etc/ssl/certs/server.crt > ssl_private_key=/etc/ssl/private/server.key > > So I don't have a combination that allows me to do just HTTPS with my > cert/key provided as a file path. > > Thanks, > Alex > > ------------------------------ > *From:* Redouane Kachach > *Sent:* Thursday, January 16, 2025 8:00 PM > *To:* Alex Hussein-Kershaw (HE/HIM) > *Cc:* ceph-users > *Subject:* Re: [EXTERNAL] Re: Cephadm: Specifying RGW Certs > & Keys By Filepath > > That's strange... in the code I can see that when rgw_frontend_port it's > used so I can't see why you get port=80 ... can you plz post your spec? > > > > On Thu, Jan 16, 2025 at 7:09 PM Alex Hussein-Kershaw (HE/HIM) < > alexhus@xxxxxxxxxxxxx> wrote: > > Oh actually I have spoke to soon. That does work, but it also exposes port > HTTP over port 80. 🙁 > > beast port=80 ssl_port=7480 ssl_certificate=/etc/ssl/certs/server.crt > ssl_private_key=/etc/ssl/private/server.key > > ------------------------------ > *From:* Alex Hussein-Kershaw (HE/HIM) > *Sent:* Thursday, January 16, 2025 5:59 PM > *To:* Redouane Kachach > *Cc:* ceph-users > *Subject:* Re: [EXTERNAL] Re: Cephadm: Specifying RGW Certs > & Keys By Filepath > > Amazing. How did I miss that. > > Dropping "ssl: true" and adding "ssl_port=1234" to the > rgw_frontend_extra_args values has me sorted. > > Many thanks! > > > ------------------------------ > *From:* Redouane Kachach > *Sent:* Thursday, January 16, 2025 4:39 PM > *To:* Alex Hussein-Kershaw (HE/HIM) > *Cc:* ceph-users > *Subject:* [EXTERNAL] Re: Cephadm: Specifying RGW Certs & > Keys By Filepath > > You are getting the double option because "ssl: true" ... try to disable > ssl since you are passing the arguments and certificates by hand! > > Another option is to have cephadm generate the certificates for you by > setting the `generate_cert` field in the spec to true. But I'm not sure if > that works for your environment or not ... > > Best, > Redo. > > On Thu, Jan 16, 2025 at 4:37 PM Alex Hussein-Kershaw (HE/HIM) < > alexhus@xxxxxxxxxxxxx> wrote: > > Hi Folks, > > Looking for some advice on RGW service specs and Cephadm. I've read the > docs here: > RGW Service — Ceph Documentation< > https://docs.ceph.com/en/reef/cephadm/services/rgw/> > > Using a service spec I can deploy a RGW: > > service_type: rgw > service_id: '25069123' > service_name: rgw.25069123 > placement: > hosts: > - raynor-sc-1 > extra_container_args: > - -v > - /etc/pki:/etc/pki:ro > - -v > - /etc/ssl/:/etc/ssl:ro > spec: > rgw_frontend_port: 7480 > rgw_realm: geored_realm > rgw_zone: siteA > rgw_zonegroup: geored_zg > ssl: true > rgw_frontend_extra_args: > - "ssl_certificate=/etc/ssl/certs/server.crt" > - "ssl_private_key=/etc/ssl/private/server.key" > > But it fails to start because my "rgw_frontends" config ends up as this. > Notice the two entries of "ssl_certificate". > > beast ssl_port=7480 ssl_certificate=config://rgw/cert/rgw.25069123 > ssl_certificate=/etc/ssl/certs/server.crt > ssl_private_key=/etc/ssl/private/server.key > > That causes it to fail to start: > > debug 2025-01-16T15:26:54.219+0000 7f65bcfdc840 0 deferred set uid:gid to > 167:167 (ceph:ceph) > debug 2025-01-16T15:26:54.219+0000 7f65bcfdc840 0 ceph version 19.2.0 > (16063ff2022298c9300e49a547a16ffda59baf13) squid (stable), process radosgw, > pid 7 > debug 2025-01-16T15:26:54.219+0000 7f65bcfdc840 0 framework: beast > debug 2025-01-16T15:26:54.219+0000 7f65bcfdc840 0 framework conf key: > ssl_port, val: 7480 > debug 2025-01-16T15:26:54.219+0000 7f65bcfdc840 0 framework conf key: > ssl_certificate, val: config://rgw/cert/rgw.25069123 > debug 2025-01-16T15:26:54.219+0000 7f65bcfdc840 0 framework conf key: > ssl_certificate, val: /etc/ssl/certs/server.crt > debug 2025-01-16T15:26:54.219+0000 7f65bcfdc840 0 framework conf key: > ssl_private_key, val: /etc/ssl/private/server.key > debug 2025-01-16T15:26:54.520+0000 7f65bcfdc840 -1 LDAP not started since > no server URIs were provided in the configuration. > debug 2025-01-16T15:26:54.531+0000 7f65bcfdc840 0 framework: beast > debug 2025-01-16T15:26:54.531+0000 7f65bcfdc840 0 framework conf key: > ssl_certificate, val: config://rgw/cert/$realm/$zone.crt > debug 2025-01-16T15:26:54.531+0000 7f65bcfdc840 0 framework conf key: > ssl_private_key, val: config://rgw/cert/$realm/$zone.key > debug 2025-01-16T15:26:54.597+0000 7f65bcfdc840 0 starting handler: beast > debug 2025-01-16T15:26:54.601+0000 7f65bcfdc840 -1 ssl_certificate was not > found: rgw/cert/rgw.25069123 > debug 2025-01-16T15:26:54.602+0000 7f65bcfdc840 -1 no ssl_certificate > configured for ssl_port > debug 2025-01-16T15:26:54.602+0000 7f65bcfdc840 -1 ERROR: failed > initializing frontend > debug 2025-01-16T15:26:54.602+0000 7f65bcfdc840 -1 ERROR: initialize > frontend fail, r = 22 > > Correcting that config manually: > > $ ceph config set client.rgw.25069123.raynor-sc-1.qiatgr rgw_frontends > "beast ssl_port=7480 ssl_certificate=/etc/ssl/certs/server.crt > ssl_private_key=/etc/ssl/private/server.key" > > Then allows the RGW to start: > > debug 2025-01-16T15:28:17.391+0000 7fba6e85a840 0 deferred set uid:gid to > 167:167 (ceph:ceph) > debug 2025-01-16T15:28:17.391+0000 7fba6e85a840 0 ceph version 19.2.0 > (16063ff2022298c9300e49a547a16ffda59baf13) squid (stable), process radosgw, > pid 7 > debug 2025-01-16T15:28:17.391+0000 7fba6e85a840 0 framework: beast > debug 2025-01-16T15:28:17.391+0000 7fba6e85a840 0 framework conf key: > ssl_port, val: 7480 > debug 2025-01-16T15:28:17.391+0000 7fba6e85a840 0 framework conf key: > ssl_certificate, val: /etc/ssl/certs/server.crt > debug 2025-01-16T15:28:17.391+0000 7fba6e85a840 0 framework conf key: > ssl_private_key, val: /etc/ssl/private/server.key > debug 2025-01-16T15:28:17.767+0000 7fba6e85a840 -1 LDAP not started since > no server URIs were provided in the configuration. > debug 2025-01-16T15:28:17.781+0000 7fba6e85a840 0 framework: beast > debug 2025-01-16T15:28:17.781+0000 7fba6e85a840 0 framework conf key: > ssl_certificate, val: config://rgw/cert/$realm/$zone.crt > debug 2025-01-16T15:28:17.781+0000 7fba6e85a840 0 framework conf key: > ssl_private_key, val: config://rgw/cert/$realm/$zone.key > debug 2025-01-16T15:28:17.860+0000 7fba6e85a840 0 starting handler: beast > debug 2025-01-16T15:28:17.908+0000 7fba6e85a840 0 set uid:gid to 167:167 > (ceph:ceph) > debug 2025-01-16T15:28:17.912+0000 7fba6e85a840 1 mgrc > service_daemon_register rgw.643295 metadata > {arch=x86_64,ceph_release=squid,ceph_version=ceph version 19.2.0 > (16063ff2022298c9300e49a547a16ffda59baf13) squid > (stable),ceph_version_short=19.2.0,container_hostname=raynor-sc-1,container_image=ceph/squid:v19.2.0,cpu=Intel(R) > Xeon(R) CPU E5-2683 v3 @ 2.00GHz,distro=centos,distro_description=CentOS > Stream 9,distro_version=9,frontend_config#0=beast ssl_port=7480 > ssl_certificate=/etc/ssl/certs/server.crt > ssl_private_key=/etc/ssl/private/server.key,frontend_type#0=beast,hostname=raynor-sc-1,id=25069123.raynor-sc-1.qiatgr,kernel_description=#1 > SMP Thu Dec 19 20:58:14 EST > 2024,kernel_version=5.4.288-1.el8.elrepo.x86_64,mem_swap_kb=0,mem_total_kb=12251748,num_handles=1,os=Linux,pid=7,realm_id=c99973d7-27cb-4abd-8391-bfbe4fa928be,realm_name=geored_realm,zone_id=30816d1d-e315-471b-bc84-6cdc4f6f1408,zone_name=siteA,zonegroup_id=c109a213-6638-4028-b21b-8bc9fcb49cd3,zonegroup_name=geored_zg} > > Specifying the certificate by file path I think is the right choice for me > as: > > * > I don't have a domain, I have a per host certificate covering the IP of > the host. Each is different, signed by a self-signed trusted CA. > * > I don't want to put the hardcoded key into the service spec, as suggested > in the docs, as it appears in the mgr logs. My org policy (sensibly) > doesn't allow for secrets to appear in logs. > * > It's just generally more hassle to manage keys on a per VM basis, rather > than a static file path. > * > It will also allow me to not manage RGW services on a per host basis, but > use the same spec for all and a count=1 placement, which would be lovely. > > So my question - is there a way around this such that I can specify the > key and certificate by file path and not have to manually change the config > to make it work? > > Thanks, > Alex > > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |