This [*] is my ceph.conf
10.70.42.9 is the public address
And it is indeed the IP used by the MON daemon:
[root@c-mon-02 ~]# netstat -anp | grep 6789
tcp 0 0 10.70.42.9:6789 0.0.0.0:* LISTEN 3835/ceph-mon
tcp 0 0 10.70.42.9:33592 10.70.42.10:6789 ESTABLISHED 3835/ceph-mon
tcp 0 0 10.70.42.9:41786 10.70.42.8:6789 ESTABLISHED 3835/ceph-mon
tcp 106008 0 10.70.42.9:33210 10.70.42.10:6789 CLOSE_WAIT 1162/ceph-mgr
tcp 100370 0 10.70.42.9:33218 10.70.42.10:6789 CLOSE_WAIT 1162/ceph-mgr
tcp 0 0 10.70.42.9:33578 10.70.42.10:6789 ESTABLISHED 1162/ceph-mgr
But the command:
/usr/bin/ceph --connect-timeout=25 --cluster=ceph --name mon. --keyring=/var/lib/ceph/mon/ceph-c-mon-02/keyring auth get client.admin
exported keyring for client.admin
fails with c-mon-02 resolved to the management IP.
As wokaround I can add in /etc/hosts the mapping with the public address:
10.70.42.9 c-mon-02
but I wonder if this is the expected behavior
Cheers, Massimo
[*]
[global]
fsid = 7a8cb8ff-562b-47da-a6aa-507136587dcf
public network = 10.70.42.0/24
cluster network = 10.69.42.0/24
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
osd pool default size = 3 # Write an object 3 times.
osd pool default min size = 2
osd pool default pg num = 128
osd pool default pgp num = 128
[mon]
mon host = c-mon-01, c-mon-02, c-mon-03
[mon.c-mon-01]
mon addr = 10.70.42.10:6789
host = c-mon-01
[mon.c-mon-02]
mon addr = 10.70.42.9:6789
host = c-mon-02
[mon.c-mon-03]
mon addr = 10.70.42.8:6789
host = c-mon-03
[osd]
osd mount options xfs = rw,noatime,inode64,logbufs=8,logbsize=256k
On Thu, May 10, 2018 at 1:12 PM, Paul Emmerich <paul.emmerich@xxxxxxxx> wrote:
check ceph.conf, it controls to which mon IP the client tries to connect.2018-05-10 12:57 GMT+02:00 Massimo Sgaravatto <massimo.sgaravatto@xxxxxxxxx>: I configured the "public network" attribute in the ceph configuration file.But it looks like to me that in the "auth get client.admin" command [*] issued by ceph-deploy the address of the management network is used (I guess because c-mon-02 gets resolved to the IP management address)Cheers, Massimo[*]/usr/bin/ceph --connect-timeout=25 --cluster=ceph --name mon. --keyring=/var/lib/ceph/mon/ceph-c-mon-02/keyring auth get client.admin On Thu, May 10, 2018 at 12:49 PM, Paul Emmerich <paul.emmerich@xxxxxxxx> wrote:Monitors can use only exactly one IP address. ceph-deploy uses some heuristics
based on hostname resolution and ceph public addr configuration to guess which
one to use during setup. (Which I've always found to be a quite annoying feature.)The mon's IP must be reachable from all ceph daemons and clients, so it should beon your "public" network. Changing the IP of a mon is possible but annoying, it isoften easier to remove and then re-add with a new IP (if possible):
http://docs.ceph.com/docs/master/rados/operations/add-or-rm- mons/#changing-a-monitor-s-ip- address Paul2018-05-10 12:36 GMT+02:00 Massimo Sgaravatto <massimo.sgaravatto@xxxxxxxxx>: ______________________________I have a ceph cluster that I manually deployed, and now I am trying to see if I can use ceph-deploy to deploy new nodes (in particular the object gw).The network configuration is the following:Each MON node has two network IP: one on a "management network" (not used for ceph related stuff) and one on a "public network",The MON daemon listens to on the pub networkEach OSD node has three network IPs: one on a "management network" (not used for ceph related stuff), one on a "public network" and the third one is an internal network to be used as ceph cluster network (for ceph internal traffic: replication, recovery, etc)Name resolution works, but names are resolved to the IP address of the management network.And it looks like this is a problem. E.g. the following command (used in ceph-deploy gatherkeys) issued on a MON host (c-mon-02) doesn't work:/usr/bin/ceph --verbose --connect-timeout=25 --cluster=ceph --name mon. --keyring=/var/lib/ceph/mon/ceph-c-mon-02/keyring auth get client.admin unless I change the name resolution of c-mon-02 to the public addressIs it a requirement (at least for ceph-deploy) that the name of each node of the ceph cluster must be resolved to the public IP address ?Thanks, Massimo_________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
----
Paul Emmerich
Looking for help with your Ceph cluster? Contact us at https://croit.io
croit GmbH
Freseniusstr. 31h
81247 München
www.croit.io
Tel: +49 89 1896585 90
----
Paul Emmerich
Looking for help with your Ceph cluster? Contact us at https://croit.io
croit GmbH
Freseniusstr. 31h
81247 München
www.croit.io
Tel: +49 89 1896585 90
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
