Hi, We have a problem with deloing radosgw vi cephadm. We have a Ceph cluster with 3 nodes deployed via cephadm. Pool creation, cephfs and block storage are working. ceph version 17.2.1 (ec95624474b1871a821a912b8c3af68f8f8e7aa1) quincy (stable) The service specs is like this for the rgw: --- service_type: rgw service_id: rgw placement: count: 3 label: "rgw" --- service_type: ingress service_id: rgw.rgw placement: count: 3 label: "ingress" spec: backend_service: rgw.rgw virtual_ip: [IPV6] virtual_interface_networks: [IPV6 CIDR] frontend_port: 8080 monitor_port: 1967 The error I get in the logfiles: 0 deferred set uid:gid to 167:167 (ceph:ceph) 0 ceph version 17.2.1 (ec95624474b1871a821a912b8c3af68f8f8e7aa1) quincy (stable), process radosgw, pid 2 0 framework: beast 0 framework conf key: port, val: 80 1 radosgw_Main not setting numa affinity 1 rgw_d3n: rgw_d3n_l1_local_datacache_enabled=0 1 D3N datacache enabled: 0 0 rgw main: rgw_init_ioctx ERROR: librados::Rados::pool_create returned (34) Numerical result out of range (this can be due to a pool or placement group misconfiguration, e.g. pg_num < pgp_num or mon_max_pg_per_osd exceeded) 0 rgw main: failed reading realm info: ret -34 (34) Numerical result out of range 0 rgw main: ERROR: failed to start notify service ((34) Numerical result out of range 0 rgw main: ERROR: failed to init services (ret=(34) Numerical result out of range) -1 Couldn't init storage provider (RADOS) I have for testing set the pg_num and pgp_num to 16 and the mon_max_pg_per_osd to 1000 and still getting the same error. I have also tried creating the rgw with ceph command, same error. Pool creation is working, I created multiple other pools and there was no problem. Thanks for any help. Best Robert The 5 fails services are 3 from the rgw and 2 haproxy for the rgw, there is only one running: ceph -s cluster: id: 40ddf health: HEALTH_WARN 5 failed cephadm daemon(s) services: mon: 3 daemons, quorum ceph-01,ceph-02,ceph-03 (age 4d) mgr: ceph-01.hbvyqi(active, since 4d), standbys: ceph-02.pqtxbv mds: 1/1 daemons up, 3 standby osd: 6 osds: 6 up (since 4d), 6 in (since 4d) data: volumes: 1/1 healthy pools: 5 pools, 65 pgs objects: 87 objects, 170 MiB usage: 1.4 GiB used, 19 TiB / 19 TiB avail pgs: 65 active+clean _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx