You are getting the double option because "ssl: true" ... try to disable ssl since you are passing the arguments and certificates by hand! Another option is to have cephadm generate the certificates for you by setting the `generate_cert` field in the spec to true. But I'm not sure if that works for your environment or not ... Best, Redo. On Thu, Jan 16, 2025 at 4:37 PM Alex Hussein-Kershaw (HE/HIM) < alexhus@xxxxxxxxxxxxx> wrote: > Hi Folks, > > Looking for some advice on RGW service specs and Cephadm. I've read the > docs here: > RGW Service — Ceph Documentation< > https://docs.ceph.com/en/reef/cephadm/services/rgw/> > > Using a service spec I can deploy a RGW: > > service_type: rgw > service_id: '25069123' > service_name: rgw.25069123 > placement: > hosts: > - raynor-sc-1 > extra_container_args: > - -v > - /etc/pki:/etc/pki:ro > - -v > - /etc/ssl/:/etc/ssl:ro > spec: > rgw_frontend_port: 7480 > rgw_realm: geored_realm > rgw_zone: siteA > rgw_zonegroup: geored_zg > ssl: true > rgw_frontend_extra_args: > - "ssl_certificate=/etc/ssl/certs/server.crt" > - "ssl_private_key=/etc/ssl/private/server.key" > > But it fails to start because my "rgw_frontends" config ends up as this. > Notice the two entries of "ssl_certificate". > > beast ssl_port=7480 ssl_certificate=config://rgw/cert/rgw.25069123 > ssl_certificate=/etc/ssl/certs/server.crt > ssl_private_key=/etc/ssl/private/server.key > > That causes it to fail to start: > > debug 2025-01-16T15:26:54.219+0000 7f65bcfdc840 0 deferred set uid:gid to > 167:167 (ceph:ceph) > debug 2025-01-16T15:26:54.219+0000 7f65bcfdc840 0 ceph version 19.2.0 > (16063ff2022298c9300e49a547a16ffda59baf13) squid (stable), process radosgw, > pid 7 > debug 2025-01-16T15:26:54.219+0000 7f65bcfdc840 0 framework: beast > debug 2025-01-16T15:26:54.219+0000 7f65bcfdc840 0 framework conf key: > ssl_port, val: 7480 > debug 2025-01-16T15:26:54.219+0000 7f65bcfdc840 0 framework conf key: > ssl_certificate, val: config://rgw/cert/rgw.25069123 > debug 2025-01-16T15:26:54.219+0000 7f65bcfdc840 0 framework conf key: > ssl_certificate, val: /etc/ssl/certs/server.crt > debug 2025-01-16T15:26:54.219+0000 7f65bcfdc840 0 framework conf key: > ssl_private_key, val: /etc/ssl/private/server.key > debug 2025-01-16T15:26:54.520+0000 7f65bcfdc840 -1 LDAP not started since > no server URIs were provided in the configuration. > debug 2025-01-16T15:26:54.531+0000 7f65bcfdc840 0 framework: beast > debug 2025-01-16T15:26:54.531+0000 7f65bcfdc840 0 framework conf key: > ssl_certificate, val: config://rgw/cert/$realm/$zone.crt > debug 2025-01-16T15:26:54.531+0000 7f65bcfdc840 0 framework conf key: > ssl_private_key, val: config://rgw/cert/$realm/$zone.key > debug 2025-01-16T15:26:54.597+0000 7f65bcfdc840 0 starting handler: beast > debug 2025-01-16T15:26:54.601+0000 7f65bcfdc840 -1 ssl_certificate was not > found: rgw/cert/rgw.25069123 > debug 2025-01-16T15:26:54.602+0000 7f65bcfdc840 -1 no ssl_certificate > configured for ssl_port > debug 2025-01-16T15:26:54.602+0000 7f65bcfdc840 -1 ERROR: failed > initializing frontend > debug 2025-01-16T15:26:54.602+0000 7f65bcfdc840 -1 ERROR: initialize > frontend fail, r = 22 > > Correcting that config manually: > > $ ceph config set client.rgw.25069123.raynor-sc-1.qiatgr rgw_frontends > "beast ssl_port=7480 ssl_certificate=/etc/ssl/certs/server.crt > ssl_private_key=/etc/ssl/private/server.key" > > Then allows the RGW to start: > > debug 2025-01-16T15:28:17.391+0000 7fba6e85a840 0 deferred set uid:gid to > 167:167 (ceph:ceph) > debug 2025-01-16T15:28:17.391+0000 7fba6e85a840 0 ceph version 19.2.0 > (16063ff2022298c9300e49a547a16ffda59baf13) squid (stable), process radosgw, > pid 7 > debug 2025-01-16T15:28:17.391+0000 7fba6e85a840 0 framework: beast > debug 2025-01-16T15:28:17.391+0000 7fba6e85a840 0 framework conf key: > ssl_port, val: 7480 > debug 2025-01-16T15:28:17.391+0000 7fba6e85a840 0 framework conf key: > ssl_certificate, val: /etc/ssl/certs/server.crt > debug 2025-01-16T15:28:17.391+0000 7fba6e85a840 0 framework conf key: > ssl_private_key, val: /etc/ssl/private/server.key > debug 2025-01-16T15:28:17.767+0000 7fba6e85a840 -1 LDAP not started since > no server URIs were provided in the configuration. > debug 2025-01-16T15:28:17.781+0000 7fba6e85a840 0 framework: beast > debug 2025-01-16T15:28:17.781+0000 7fba6e85a840 0 framework conf key: > ssl_certificate, val: config://rgw/cert/$realm/$zone.crt > debug 2025-01-16T15:28:17.781+0000 7fba6e85a840 0 framework conf key: > ssl_private_key, val: config://rgw/cert/$realm/$zone.key > debug 2025-01-16T15:28:17.860+0000 7fba6e85a840 0 starting handler: beast > debug 2025-01-16T15:28:17.908+0000 7fba6e85a840 0 set uid:gid to 167:167 > (ceph:ceph) > debug 2025-01-16T15:28:17.912+0000 7fba6e85a840 1 mgrc > service_daemon_register rgw.643295 metadata > {arch=x86_64,ceph_release=squid,ceph_version=ceph version 19.2.0 > (16063ff2022298c9300e49a547a16ffda59baf13) squid > (stable),ceph_version_short=19.2.0,container_hostname=raynor-sc-1,container_image=ceph/squid:v19.2.0,cpu=Intel(R) > Xeon(R) CPU E5-2683 v3 @ 2.00GHz,distro=centos,distro_description=CentOS > Stream 9,distro_version=9,frontend_config#0=beast ssl_port=7480 > ssl_certificate=/etc/ssl/certs/server.crt > ssl_private_key=/etc/ssl/private/server.key,frontend_type#0=beast,hostname=raynor-sc-1,id=25069123.raynor-sc-1.qiatgr,kernel_description=#1 > SMP Thu Dec 19 20:58:14 EST > 2024,kernel_version=5.4.288-1.el8.elrepo.x86_64,mem_swap_kb=0,mem_total_kb=12251748,num_handles=1,os=Linux,pid=7,realm_id=c99973d7-27cb-4abd-8391-bfbe4fa928be,realm_name=geored_realm,zone_id=30816d1d-e315-471b-bc84-6cdc4f6f1408,zone_name=siteA,zonegroup_id=c109a213-6638-4028-b21b-8bc9fcb49cd3,zonegroup_name=geored_zg} > > Specifying the certificate by file path I think is the right choice for me > as: > > * > I don't have a domain, I have a per host certificate covering the IP of > the host. Each is different, signed by a self-signed trusted CA. > * > I don't want to put the hardcoded key into the service spec, as suggested > in the docs, as it appears in the mgr logs. My org policy (sensibly) > doesn't allow for secrets to appear in logs. > * > It's just generally more hassle to manage keys on a per VM basis, rather > than a static file path. > * > It will also allow me to not manage RGW services on a per host basis, but > use the same spec for all and a count=1 placement, which would be lovely. > > So my question - is there a way around this such that I can specify the > key and certificate by file path and not have to manually change the config > to make it work? > > Thanks, > Alex > > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |