Nathan; I'm not an expert on firewalld, but shouldn't you have a list of open ports? ports: ????? Here's the configuration on my test cluster: public (active) target: default icmp-block-inversion: no interfaces: bond0 sources: services: ssh dhcpv6-client ports: 6789/tcp 3300/tcp 6800-7300/tcp 8443/tcp protocols: masquerade: no forward-ports: source-ports: icmp-blocks: rich rules: trusted (active) target: ACCEPT icmp-block-inversion: no interfaces: bond1 sources: services: ports: 6789/tcp 3300/tcp 6800-7300/tcp 8443/tcp protocols: masquerade: no forward-ports: source-ports: icmp-blocks: rich rules: I use interfaces as selectors, but would think source selectors would work the same. You might start by adding the MON ports to the firewall on the MONs: firewall-cmd --zone=public --add-port=6789/tcp --permanent firewall-cmd --zone=public --add-port=3300/tcp --permanent firewall-cmd --reload Thank you, Dominic L. Hilsbos, MBA Director – Information Technology Perform Air International Inc. DHilsbos@xxxxxxxxxxxxxx www.PerformAir.com From: ceph-users [mailto:ceph-users-bounces@xxxxxxxxxxxxxx] On Behalf Of Nathan Harper Sent: Thursday, July 25, 2019 2:08 PM To: ceph-users@xxxxxxxx Subject: [Disarmed] Re: ceph-ansible firewalld blocking ceph comms This is a new issue to us, and did not have the same problem running the same activity on our test system. Regards, Nathan On 25 Jul 2019, at 22:00, solarflow99 <solarflow99@xxxxxxxxx> wrote: I used ceph-ansible just fine, never had this problem. On Thu, Jul 25, 2019 at 1:31 PM Nathan Harper <nathan.harper@xxxxxxxxxxx> wrote: Hi all, We've run into a strange issue with one of our clusters managed with ceph-ansible. We're adding some RGW nodes to our cluster, and so re-ran site.yml against the cluster. The new RGWs added successfully, but.... When we did, we started to get slow requests, effectively across the whole cluster. Quickly we realised that the firewall was now (apparently) blocking Ceph communications. I say apparently, because the config looks correct: [root@osdsrv05 ~]# firewall-cmd --list-all public (active) target: default icmp-block-inversion: no interfaces: sources: MailScanner has detected a possible fraud attempt from "172.20.22.0" claiming to be 172.20.22.0/24 MailScanner has detected a possible fraud attempt from "172.20.23.0" claiming to be 172.20.23.0/24 services: ssh dhcpv6-client ceph ports: protocols: masquerade: no forward-ports: source-ports: icmp-blocks: rich rules: If we drop the firewall everything goes back healthy. All the clients (Openstack cinder) are on the 172.20.22.0 network (172.20.23.0 is the replication network). Has anyone seen this? -- Nathan Harper // IT Systems Lead _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |