I see, unfortunately I can't see an easy way to avoid that. With the current code you will get either port= or ssl_port= depending on whether you enable ssl or not. At the end of the day the result get written in the configuration variable rgw_frontends (ceph config ls | grep rgw_frontends). I'll recommend opening a ticket with a feature request explaining your use case and why the current behavior doesn't meet your needs. Best, Redo. On Fri, Jan 17, 2025 at 9:51 AM Alex Hussein-Kershaw (HE/HIM) < alexhus@xxxxxxxxxxxxx> wrote: > Moring Redo, > > I had dropped the "rgw_frontend_port" as I was providing it via > "ssl_port". However I tested with it back in also: > > service_type: rgw > service_id: rgw > service_name: rgw.rgw > placement: > count_per_host: 1 > host_pattern: '*' > extra_container_args: > - -v > - /etc/pki:/etc/pki:ro > - -v > - /etc/ssl/:/etc/ssl:ro > spec: > rgw_frontend_port: 7480 > rgw_frontend_extra_args: > - ssl_port=7480 > - ssl_certificate=/etc/ssl/certs/server.crt > - ssl_private_key=/etc/ssl/private/server.key > rgw_realm: geored_realm > rgw_zone: siteA > rgw_zonegroup: geored_zg > > Results in config: > > beast port=7480 ssl_port=7480 ssl_certificate=/etc/ssl/certs/server.crt > ssl_private_key=/etc/ssl/private/server.key > > > ...and I then get errors in logs because it's trying to bind to the same > port twice. > > failed to bind address 0.0.0.0:7480: Address already in use > > Dropping the "ssl_port" just brings it up as a HTTP endpoint. > > beast port=7480 ssl_certificate=/etc/ssl/certs/server.crt > ssl_private_key=/etc/ssl/private/server.key > > So I don't have a combination that allows me to do just HTTPS with my > cert/key provided as a file path. > > Thanks, > Alex > > ------------------------------ > *From:* Redouane Kachach > *Sent:* Thursday, January 16, 2025 8:00 PM > *To:* Alex Hussein-Kershaw (HE/HIM) > *Cc:* ceph-users > *Subject:* Re: [EXTERNAL] Re: Cephadm: Specifying RGW Certs > & Keys By Filepath > > That's strange... in the code I can see that when rgw_frontend_port it's > used so I can't see why you get port=80 ... can you plz post your spec? > > > > On Thu, Jan 16, 2025 at 7:09 PM Alex Hussein-Kershaw (HE/HIM) < > alexhus@xxxxxxxxxxxxx> wrote: > > Oh actually I have spoke to soon. That does work, but it also exposes port > HTTP over port 80. 🙁 > > beast port=80 ssl_port=7480 ssl_certificate=/etc/ssl/certs/server.crt > ssl_private_key=/etc/ssl/private/server.key > > ------------------------------ > *From:* Alex Hussein-Kershaw (HE/HIM) > *Sent:* Thursday, January 16, 2025 5:59 PM > *To:* Redouane Kachach > *Cc:* ceph-users > *Subject:* Re: [EXTERNAL] Re: Cephadm: Specifying RGW Certs > & Keys By Filepath > > Amazing. How did I miss that. > > Dropping "ssl: true" and adding "ssl_port=1234" to the > rgw_frontend_extra_args values has me sorted. > > Many thanks! > > > ------------------------------ > *From:* Redouane Kachach > *Sent:* Thursday, January 16, 2025 4:39 PM > *To:* Alex Hussein-Kershaw (HE/HIM) > *Cc:* ceph-users > *Subject:* [EXTERNAL] Re: Cephadm: Specifying RGW Certs & > Keys By Filepath > > You are getting the double option because "ssl: true" ... try to disable > ssl since you are passing the arguments and certificates by hand! > > Another option is to have cephadm generate the certificates for you by > setting the `generate_cert` field in the spec to true. But I'm not sure if > that works for your environment or not ... > > Best, > Redo. > > On Thu, Jan 16, 2025 at 4:37 PM Alex Hussein-Kershaw (HE/HIM) < > alexhus@xxxxxxxxxxxxx> wrote: > > Hi Folks, > > Looking for some advice on RGW service specs and Cephadm. I've read the > docs here: > RGW Service — Ceph Documentation< > https://docs.ceph.com/en/reef/cephadm/services/rgw/> > > Using a service spec I can deploy a RGW: > > service_type: rgw > service_id: '25069123' > service_name: rgw.25069123 > placement: > hosts: > - raynor-sc-1 > extra_container_args: > - -v > - /etc/pki:/etc/pki:ro > - -v > - /etc/ssl/:/etc/ssl:ro > spec: > rgw_frontend_port: 7480 > rgw_realm: geored_realm > rgw_zone: siteA > rgw_zonegroup: geored_zg > ssl: true > rgw_frontend_extra_args: > - "ssl_certificate=/etc/ssl/certs/server.crt" > - "ssl_private_key=/etc/ssl/private/server.key" > > But it fails to start because my "rgw_frontends" config ends up as this. > Notice the two entries of "ssl_certificate". > > beast ssl_port=7480 ssl_certificate=config://rgw/cert/rgw.25069123 > ssl_certificate=/etc/ssl/certs/server.crt > ssl_private_key=/etc/ssl/private/server.key > > That causes it to fail to start: > > debug 2025-01-16T15:26:54.219+0000 7f65bcfdc840 0 deferred set uid:gid to > 167:167 (ceph:ceph) > debug 2025-01-16T15:26:54.219+0000 7f65bcfdc840 0 ceph version 19.2.0 > (16063ff2022298c9300e49a547a16ffda59baf13) squid (stable), process radosgw, > pid 7 > debug 2025-01-16T15:26:54.219+0000 7f65bcfdc840 0 framework: beast > debug 2025-01-16T15:26:54.219+0000 7f65bcfdc840 0 framework conf key: > ssl_port, val: 7480 > debug 2025-01-16T15:26:54.219+0000 7f65bcfdc840 0 framework conf key: > ssl_certificate, val: config://rgw/cert/rgw.25069123 > debug 2025-01-16T15:26:54.219+0000 7f65bcfdc840 0 framework conf key: > ssl_certificate, val: /etc/ssl/certs/server.crt > debug 2025-01-16T15:26:54.219+0000 7f65bcfdc840 0 framework conf key: > ssl_private_key, val: /etc/ssl/private/server.key > debug 2025-01-16T15:26:54.520+0000 7f65bcfdc840 -1 LDAP not started since > no server URIs were provided in the configuration. > debug 2025-01-16T15:26:54.531+0000 7f65bcfdc840 0 framework: beast > debug 2025-01-16T15:26:54.531+0000 7f65bcfdc840 0 framework conf key: > ssl_certificate, val: config://rgw/cert/$realm/$zone.crt > debug 2025-01-16T15:26:54.531+0000 7f65bcfdc840 0 framework conf key: > ssl_private_key, val: config://rgw/cert/$realm/$zone.key > debug 2025-01-16T15:26:54.597+0000 7f65bcfdc840 0 starting handler: beast > debug 2025-01-16T15:26:54.601+0000 7f65bcfdc840 -1 ssl_certificate was not > found: rgw/cert/rgw.25069123 > debug 2025-01-16T15:26:54.602+0000 7f65bcfdc840 -1 no ssl_certificate > configured for ssl_port > debug 2025-01-16T15:26:54.602+0000 7f65bcfdc840 -1 ERROR: failed > initializing frontend > debug 2025-01-16T15:26:54.602+0000 7f65bcfdc840 -1 ERROR: initialize > frontend fail, r = 22 > > Correcting that config manually: > > $ ceph config set client.rgw.25069123.raynor-sc-1.qiatgr rgw_frontends > "beast ssl_port=7480 ssl_certificate=/etc/ssl/certs/server.crt > ssl_private_key=/etc/ssl/private/server.key" > > Then allows the RGW to start: > > debug 2025-01-16T15:28:17.391+0000 7fba6e85a840 0 deferred set uid:gid to > 167:167 (ceph:ceph) > debug 2025-01-16T15:28:17.391+0000 7fba6e85a840 0 ceph version 19.2.0 > (16063ff2022298c9300e49a547a16ffda59baf13) squid (stable), process radosgw, > pid 7 > debug 2025-01-16T15:28:17.391+0000 7fba6e85a840 0 framework: beast > debug 2025-01-16T15:28:17.391+0000 7fba6e85a840 0 framework conf key: > ssl_port, val: 7480 > debug 2025-01-16T15:28:17.391+0000 7fba6e85a840 0 framework conf key: > ssl_certificate, val: /etc/ssl/certs/server.crt > debug 2025-01-16T15:28:17.391+0000 7fba6e85a840 0 framework conf key: > ssl_private_key, val: /etc/ssl/private/server.key > debug 2025-01-16T15:28:17.767+0000 7fba6e85a840 -1 LDAP not started since > no server URIs were provided in the configuration. > debug 2025-01-16T15:28:17.781+0000 7fba6e85a840 0 framework: beast > debug 2025-01-16T15:28:17.781+0000 7fba6e85a840 0 framework conf key: > ssl_certificate, val: config://rgw/cert/$realm/$zone.crt > debug 2025-01-16T15:28:17.781+0000 7fba6e85a840 0 framework conf key: > ssl_private_key, val: config://rgw/cert/$realm/$zone.key > debug 2025-01-16T15:28:17.860+0000 7fba6e85a840 0 starting handler: beast > debug 2025-01-16T15:28:17.908+0000 7fba6e85a840 0 set uid:gid to 167:167 > (ceph:ceph) > debug 2025-01-16T15:28:17.912+0000 7fba6e85a840 1 mgrc > service_daemon_register rgw.643295 metadata > {arch=x86_64,ceph_release=squid,ceph_version=ceph version 19.2.0 > (16063ff2022298c9300e49a547a16ffda59baf13) squid > (stable),ceph_version_short=19.2.0,container_hostname=raynor-sc-1,container_image=ceph/squid:v19.2.0,cpu=Intel(R) > Xeon(R) CPU E5-2683 v3 @ 2.00GHz,distro=centos,distro_description=CentOS > Stream 9,distro_version=9,frontend_config#0=beast ssl_port=7480 > ssl_certificate=/etc/ssl/certs/server.crt > ssl_private_key=/etc/ssl/private/server.key,frontend_type#0=beast,hostname=raynor-sc-1,id=25069123.raynor-sc-1.qiatgr,kernel_description=#1 > SMP Thu Dec 19 20:58:14 EST > 2024,kernel_version=5.4.288-1.el8.elrepo.x86_64,mem_swap_kb=0,mem_total_kb=12251748,num_handles=1,os=Linux,pid=7,realm_id=c99973d7-27cb-4abd-8391-bfbe4fa928be,realm_name=geored_realm,zone_id=30816d1d-e315-471b-bc84-6cdc4f6f1408,zone_name=siteA,zonegroup_id=c109a213-6638-4028-b21b-8bc9fcb49cd3,zonegroup_name=geored_zg} > > Specifying the certificate by file path I think is the right choice for me > as: > > * > I don't have a domain, I have a per host certificate covering the IP of > the host. Each is different, signed by a self-signed trusted CA. > * > I don't want to put the hardcoded key into the service spec, as suggested > in the docs, as it appears in the mgr logs. My org policy (sensibly) > doesn't allow for secrets to appear in logs. > * > It's just generally more hassle to manage keys on a per VM basis, rather > than a static file path. > * > It will also allow me to not manage RGW services on a per host basis, but > use the same spec for all and a count=1 placement, which would be lovely. > > So my question - is there a way around this such that I can specify the > key and certificate by file path and not have to manually change the config > to make it work? > > Thanks, > Alex > > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |