Thank you Alex, Reading the description of the tracker https://tracker.ceph.com/issues/69567 sounds like the outcome is very similar to when using the self-signed auto-generated certificates by cephadm. Wouldn't that be a viable option? I mean is there a big difference between self-signing the certificates by yourself and having cephadm doing that for you (unless your Org has strict rules around that!)? On Fri, Jan 17, 2025 at 10:19 AM Redouane Kachach <rkachach@xxxxxxxxxx> wrote: > I see, unfortunately I can't see an easy way to avoid that. With the > current code you will get either port= or ssl_port= depending on whether > you enable ssl or not. At the end of the day the result get written in the > configuration variable rgw_frontends (ceph config ls | grep rgw_frontends). > > I'll recommend opening a ticket with a feature request explaining your use > case and why the current behavior doesn't meet your needs. > > Best, > Redo. > > > > > On Fri, Jan 17, 2025 at 9:51 AM Alex Hussein-Kershaw (HE/HIM) < > alexhus@xxxxxxxxxxxxx> wrote: > >> Moring Redo, >> >> I had dropped the "rgw_frontend_port" as I was providing it via >> "ssl_port". However I tested with it back in also: >> >> service_type: rgw >> service_id: rgw >> service_name: rgw.rgw >> placement: >> count_per_host: 1 >> host_pattern: '*' >> extra_container_args: >> - -v >> - /etc/pki:/etc/pki:ro >> - -v >> - /etc/ssl/:/etc/ssl:ro >> spec: >> rgw_frontend_port: 7480 >> rgw_frontend_extra_args: >> - ssl_port=7480 >> - ssl_certificate=/etc/ssl/certs/server.crt >> - ssl_private_key=/etc/ssl/private/server.key >> rgw_realm: geored_realm >> rgw_zone: siteA >> rgw_zonegroup: geored_zg >> >> Results in config: >> >> beast port=7480 ssl_port=7480 ssl_certificate=/etc/ssl/certs/server.crt >> ssl_private_key=/etc/ssl/private/server.key >> >> >> ...and I then get errors in logs because it's trying to bind to the same >> port twice. >> >> failed to bind address 0.0.0.0:7480: Address already in use >> >> Dropping the "ssl_port" just brings it up as a HTTP endpoint. >> >> beast port=7480 ssl_certificate=/etc/ssl/certs/server.crt >> ssl_private_key=/etc/ssl/private/server.key >> >> So I don't have a combination that allows me to do just HTTPS with my >> cert/key provided as a file path. >> >> Thanks, >> Alex >> >> ------------------------------ >> *From:* Redouane Kachach >> *Sent:* Thursday, January 16, 2025 8:00 PM >> *To:* Alex Hussein-Kershaw (HE/HIM) >> *Cc:* ceph-users >> *Subject:* Re: [EXTERNAL] Re: Cephadm: Specifying RGW Certs >> & Keys By Filepath >> >> That's strange... in the code I can see that when rgw_frontend_port it's >> used so I can't see why you get port=80 ... can you plz post your spec? >> >> >> >> On Thu, Jan 16, 2025 at 7:09 PM Alex Hussein-Kershaw (HE/HIM) < >> alexhus@xxxxxxxxxxxxx> wrote: >> >> Oh actually I have spoke to soon. That does work, but it also exposes >> port HTTP over port 80. 🙁 >> >> beast port=80 ssl_port=7480 ssl_certificate=/etc/ssl/certs/server.crt >> ssl_private_key=/etc/ssl/private/server.key >> >> ------------------------------ >> *From:* Alex Hussein-Kershaw (HE/HIM) >> *Sent:* Thursday, January 16, 2025 5:59 PM >> *To:* Redouane Kachach >> *Cc:* ceph-users >> *Subject:* Re: [EXTERNAL] Re: Cephadm: Specifying RGW Certs >> & Keys By Filepath >> >> Amazing. How did I miss that. >> >> Dropping "ssl: true" and adding "ssl_port=1234" to the >> rgw_frontend_extra_args values has me sorted. >> >> Many thanks! >> >> >> ------------------------------ >> *From:* Redouane Kachach >> *Sent:* Thursday, January 16, 2025 4:39 PM >> *To:* Alex Hussein-Kershaw (HE/HIM) >> *Cc:* ceph-users >> *Subject:* [EXTERNAL] Re: Cephadm: Specifying RGW Certs & >> Keys By Filepath >> >> You are getting the double option because "ssl: true" ... try to disable >> ssl since you are passing the arguments and certificates by hand! >> >> Another option is to have cephadm generate the certificates for you by >> setting the `generate_cert` field in the spec to true. But I'm not sure if >> that works for your environment or not ... >> >> Best, >> Redo. >> >> On Thu, Jan 16, 2025 at 4:37 PM Alex Hussein-Kershaw (HE/HIM) < >> alexhus@xxxxxxxxxxxxx> wrote: >> >> Hi Folks, >> >> Looking for some advice on RGW service specs and Cephadm. I've read the >> docs here: >> RGW Service — Ceph Documentation< >> https://docs.ceph.com/en/reef/cephadm/services/rgw/> >> >> Using a service spec I can deploy a RGW: >> >> service_type: rgw >> service_id: '25069123' >> service_name: rgw.25069123 >> placement: >> hosts: >> - raynor-sc-1 >> extra_container_args: >> - -v >> - /etc/pki:/etc/pki:ro >> - -v >> - /etc/ssl/:/etc/ssl:ro >> spec: >> rgw_frontend_port: 7480 >> rgw_realm: geored_realm >> rgw_zone: siteA >> rgw_zonegroup: geored_zg >> ssl: true >> rgw_frontend_extra_args: >> - "ssl_certificate=/etc/ssl/certs/server.crt" >> - "ssl_private_key=/etc/ssl/private/server.key" >> >> But it fails to start because my "rgw_frontends" config ends up as this. >> Notice the two entries of "ssl_certificate". >> >> beast ssl_port=7480 ssl_certificate=config://rgw/cert/rgw.25069123 >> ssl_certificate=/etc/ssl/certs/server.crt >> ssl_private_key=/etc/ssl/private/server.key >> >> That causes it to fail to start: >> >> debug 2025-01-16T15:26:54.219+0000 7f65bcfdc840 0 deferred set uid:gid >> to 167:167 (ceph:ceph) >> debug 2025-01-16T15:26:54.219+0000 7f65bcfdc840 0 ceph version 19.2.0 >> (16063ff2022298c9300e49a547a16ffda59baf13) squid (stable), process radosgw, >> pid 7 >> debug 2025-01-16T15:26:54.219+0000 7f65bcfdc840 0 framework: beast >> debug 2025-01-16T15:26:54.219+0000 7f65bcfdc840 0 framework conf key: >> ssl_port, val: 7480 >> debug 2025-01-16T15:26:54.219+0000 7f65bcfdc840 0 framework conf key: >> ssl_certificate, val: config://rgw/cert/rgw.25069123 >> debug 2025-01-16T15:26:54.219+0000 7f65bcfdc840 0 framework conf key: >> ssl_certificate, val: /etc/ssl/certs/server.crt >> debug 2025-01-16T15:26:54.219+0000 7f65bcfdc840 0 framework conf key: >> ssl_private_key, val: /etc/ssl/private/server.key >> debug 2025-01-16T15:26:54.520+0000 7f65bcfdc840 -1 LDAP not started since >> no server URIs were provided in the configuration. >> debug 2025-01-16T15:26:54.531+0000 7f65bcfdc840 0 framework: beast >> debug 2025-01-16T15:26:54.531+0000 7f65bcfdc840 0 framework conf key: >> ssl_certificate, val: config://rgw/cert/$realm/$zone.crt >> debug 2025-01-16T15:26:54.531+0000 7f65bcfdc840 0 framework conf key: >> ssl_private_key, val: config://rgw/cert/$realm/$zone.key >> debug 2025-01-16T15:26:54.597+0000 7f65bcfdc840 0 starting handler: beast >> debug 2025-01-16T15:26:54.601+0000 7f65bcfdc840 -1 ssl_certificate was >> not found: rgw/cert/rgw.25069123 >> debug 2025-01-16T15:26:54.602+0000 7f65bcfdc840 -1 no ssl_certificate >> configured for ssl_port >> debug 2025-01-16T15:26:54.602+0000 7f65bcfdc840 -1 ERROR: failed >> initializing frontend >> debug 2025-01-16T15:26:54.602+0000 7f65bcfdc840 -1 ERROR: initialize >> frontend fail, r = 22 >> >> Correcting that config manually: >> >> $ ceph config set client.rgw.25069123.raynor-sc-1.qiatgr rgw_frontends >> "beast ssl_port=7480 ssl_certificate=/etc/ssl/certs/server.crt >> ssl_private_key=/etc/ssl/private/server.key" >> >> Then allows the RGW to start: >> >> debug 2025-01-16T15:28:17.391+0000 7fba6e85a840 0 deferred set uid:gid >> to 167:167 (ceph:ceph) >> debug 2025-01-16T15:28:17.391+0000 7fba6e85a840 0 ceph version 19.2.0 >> (16063ff2022298c9300e49a547a16ffda59baf13) squid (stable), process radosgw, >> pid 7 >> debug 2025-01-16T15:28:17.391+0000 7fba6e85a840 0 framework: beast >> debug 2025-01-16T15:28:17.391+0000 7fba6e85a840 0 framework conf key: >> ssl_port, val: 7480 >> debug 2025-01-16T15:28:17.391+0000 7fba6e85a840 0 framework conf key: >> ssl_certificate, val: /etc/ssl/certs/server.crt >> debug 2025-01-16T15:28:17.391+0000 7fba6e85a840 0 framework conf key: >> ssl_private_key, val: /etc/ssl/private/server.key >> debug 2025-01-16T15:28:17.767+0000 7fba6e85a840 -1 LDAP not started since >> no server URIs were provided in the configuration. >> debug 2025-01-16T15:28:17.781+0000 7fba6e85a840 0 framework: beast >> debug 2025-01-16T15:28:17.781+0000 7fba6e85a840 0 framework conf key: >> ssl_certificate, val: config://rgw/cert/$realm/$zone.crt >> debug 2025-01-16T15:28:17.781+0000 7fba6e85a840 0 framework conf key: >> ssl_private_key, val: config://rgw/cert/$realm/$zone.key >> debug 2025-01-16T15:28:17.860+0000 7fba6e85a840 0 starting handler: beast >> debug 2025-01-16T15:28:17.908+0000 7fba6e85a840 0 set uid:gid to 167:167 >> (ceph:ceph) >> debug 2025-01-16T15:28:17.912+0000 7fba6e85a840 1 mgrc >> service_daemon_register rgw.643295 metadata >> {arch=x86_64,ceph_release=squid,ceph_version=ceph version 19.2.0 >> (16063ff2022298c9300e49a547a16ffda59baf13) squid >> (stable),ceph_version_short=19.2.0,container_hostname=raynor-sc-1,container_image=ceph/squid:v19.2.0,cpu=Intel(R) >> Xeon(R) CPU E5-2683 v3 @ 2.00GHz,distro=centos,distro_description=CentOS >> Stream 9,distro_version=9,frontend_config#0=beast ssl_port=7480 >> ssl_certificate=/etc/ssl/certs/server.crt >> ssl_private_key=/etc/ssl/private/server.key,frontend_type#0=beast,hostname=raynor-sc-1,id=25069123.raynor-sc-1.qiatgr,kernel_description=#1 >> SMP Thu Dec 19 20:58:14 EST >> 2024,kernel_version=5.4.288-1.el8.elrepo.x86_64,mem_swap_kb=0,mem_total_kb=12251748,num_handles=1,os=Linux,pid=7,realm_id=c99973d7-27cb-4abd-8391-bfbe4fa928be,realm_name=geored_realm,zone_id=30816d1d-e315-471b-bc84-6cdc4f6f1408,zone_name=siteA,zonegroup_id=c109a213-6638-4028-b21b-8bc9fcb49cd3,zonegroup_name=geored_zg} >> >> Specifying the certificate by file path I think is the right choice for >> me as: >> >> * >> I don't have a domain, I have a per host certificate covering the IP of >> the host. Each is different, signed by a self-signed trusted CA. >> * >> I don't want to put the hardcoded key into the service spec, as suggested >> in the docs, as it appears in the mgr logs. My org policy (sensibly) >> doesn't allow for secrets to appear in logs. >> * >> It's just generally more hassle to manage keys on a per VM basis, rather >> than a static file path. >> * >> It will also allow me to not manage RGW services on a per host basis, but >> use the same spec for all and a count=1 placement, which would be lovely. >> >> So my question - is there a way around this such that I can specify the >> key and certificate by file path and not have to manually change the config >> to make it work? >> >> Thanks, >> Alex >> >> _______________________________________________ >> ceph-users mailing list -- ceph-users@xxxxxxx >> To unsubscribe send an email to ceph-users-leave@xxxxxxx >> >> _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |