Hello again So I have changed the network configuration. Now my Ceph is reachable from outside, this also means all osd’s of all nodes are reachable. I still have the same behaviour which is a timeout. The client can resolve all nodes with their hostnames. The mon’s are still listening on the internal network so the nat rule is still there. I have set “public bind addr” to the external ip and restarted the mon but it’s still not working. [root@testnode1 ~]# ceph config get mon.public_bind_addr WHO MASK LEVEL OPTION VALUE RO mon advanced public_bind_addr v2:[ext-addr]:0/0 * Do I have to change them somewhere else too? Thanks in advance, Simon Von: Janne Johansson [mailto:icepic.dz@xxxxxxxxx] Gesendet: 27 August 2020 20:01 An: Simon Sutter <ssutter@xxxxxxxxxxx> Betreff: Re: cephfs needs access from two networks Den tors 27 aug. 2020 kl 12:05 skrev Simon Sutter <ssutter@xxxxxxxxxxx<mailto:ssutter@xxxxxxxxxxx>>: Hello Janne Oh I missed that point. No, the client cannot talk directly to the osds. In this case it’s extremely difficult to set this up. This is an absolute requirement to be a ceph client. How is the mon telling the client, which host and port of the osd, it should connect to? The same port and ip that the ODS called into the mon with when it started up and joined the clusster. Can I have an influence on it? Well, you set the ip on the OSD hosts, and the port ranges in use for OSDs are changeable/settable, but it would not really help the above-mentioned client. Von: Janne Johansson [mailto:icepic.dz@xxxxxxxxx<mailto:icepic.dz@xxxxxxxxx>] Gesendet: 26 August 2020 15:09 An: Simon Sutter <ssutter@xxxxxxxxxxx<mailto:ssutter@xxxxxxxxxxx>> Cc: ceph-users@xxxxxxx<mailto:ceph-users@xxxxxxx> Betreff: Re: cephfs needs access from two networks Den ons 26 aug. 2020 kl 14:16 skrev Simon Sutter <ssutter@xxxxxxxxxxx<mailto:ssutter@xxxxxxxxxxx>>: Hello, So I know, the mon services can only bind to just one ip. But I have to make it accessible to two networks because internal and external servers have to mount the cephfs. The internal ip is 10.99.10.1 and the external is some public-ip. I tried nat'ing it with this: "firewall-cmd --zone=public --add-forward-port=port=6789:proto=tcp:toport=6789:toaddr=10.99.10.1 -permanent" So the nat is working, because I get a "ceph v027" (alongside with some gibberish) when I do a telnet "telnet *public-ip* 6789" But when I try to mount it, I get just a timeout: mount -vvvv -t ceph *public-ip*:6789:/testing /mnt -o name=test,secretfile=/root/ceph.client. test.key mount error 110 = Connection timed out The tcpdump also recognizes a "Ceph Connect" packet, coming from the mon. How can I get around this problem? Is there something I have missed? Any ceph client will need direct access to all OSDs involved also. Your mail doesn't really say if the cephfs-mounting client can talk to OSDs? In ceph, traffic is not shuffled via mons, mons only tell the client which OSDs it needs to talk to, then all IO goes directly from client to any involved OSD servers. -- May the most significant bit of your life be positive. -- May the most significant bit of your life be positive. _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |