Hi,
Migrated the cluster to new IP range. Regarding the MON that doesn't listen to v1 port, maybe I ran the command as mentioned in manual, but seems the part after comma doesn't work and tells the mon to listen on v2 port only.
ceph-mon -i cmon5 --public-addr v2:10.0.1.97:3300,v1:10.0.1.97:6789
MON map resumed normal with both v1 and v2 for this MON after removing the MON and add it again without running this command after mkfs, then start the services as usual.
Thanks a lot.
Rgds
/stwong
-----Original Message-----
From: ceph-users <ceph-users-bounces@xxxxxxxxxxxxxx> On Behalf Of ST Wong (ITSC)
Sent: Wednesday, July 24, 2019 2:20 PM
To: Manuel Lausch <manuel.lausch@xxxxxxxx>; ceph-users@xxxxxxxxxxxxxx
Subject: Re: Please help: change IP address of a cluster
Hi,
Thanks for your help.
I changed IP addresses of OSD nodes successfully. When changing IP address on MON, it works except that the MON only listens on v2 port 3300 after adding the MON back to the cluster. Previously the MON listens on both v1 (6789) and v2 (3300).
Besides, can't add both v1 and v2 entries manually using monmaptool like following. Only the 2nd one will get added.
monmaptool -add node1 v1:10.0.1.1:6789, v2:10.0.1.1:3330 {tmp}/{filename}
the monmap now looks like following:
min_mon_release 14 (nautilus)
0: [v2:10.0.1.92:3300/0,v1:10.0.1.92:6789/0] mon.cmon2
1: [v2:10.0.1.93:3300/0,v1:10.0.1.93:6789/0] mon.cmon2
2: [v2:10.0.1.94:3300/0,v1:10.0.1.94:6789/0] mon.cmon3
3: [v2:10.0.1.95:3300/0,v1:10.0.1.95:6789/0] mon.cmon4
4: v2:10.0.1.97:3300/0 mon.cmon5 <--- the MON being removed/added
Although it's okay to use v2 only, I'm afraid I've missed some steps and messed the cluster up. Any advice?
Thanks again.
Best Regards,
/stwong
-----Original Message-----
From: ceph-users <ceph-users-bounces@xxxxxxxxxxxxxx> On Behalf Of Manuel Lausch
Sent: Tuesday, July 23, 2019 7:32 PM
To: ceph-users@xxxxxxxxxxxxxx
Subject: Re: Please help: change IP address of a cluster
Hi,
I had to change the IPs of my cluster some time ago. The process was quite easy.
I don't understand what you mean with configuring and deleting static routes. The easies way is if the router allows (at least for the
change) all traffic between the old and the new network.
I did the following steps.
1. Add the new IP Network space separated to the "public network" line in your ceph.conf
2. OSDS: stop you OSDs on the first node. Reconfigure the host network and start your OSDs again. Repeat this for all hosts one by one
3. MON: stop and remove one mon from cluster, delete all data in /var/ceph/mon/mon.<id> reconfigure the host network. Create the new mon instance (don't forget the "mon host" entrys in your ceph.conf and your clients as well) Of course this requires at least 3 Mons in your cluster!
After 2 of 5 Mons in my cluster I added the new mon adresses to my clients and restarted them.
4. MGR: stop the mgr daemon. reconfigure the host network. Start the mgr daemon one by one
I wouldn't recomend the "messy way" to reconfigure your mons. removing and adding mons to the cluster is quite easy and in my opinion the most secure.
The complet IP change in our cluster worked without outage while the cluster was in production.
I hope I could help you.
Regards
Manuel
On Fri, 19 Jul 2019 10:22:37 +0000
"ST Wong (ITSC)" <ST@xxxxxxxxxxxxxxxx> wrote:
> Hi all,
>
> Our cluster has to change to new IP range in same VLAN: 10.0.7.0/24
> -> 10.0.18.0/23, while IP address on private network for OSDs
> remains unchanged. I wonder if we can do that in either one following
> ways:
>
> =====================
>
> 1.
>
> a. Define static route for 10.0.18.0/23 on each node
>
> b. Do it one by one:
>
> For each monitor/mgr:
>
> - remove from cluster
>
> - change IP address
>
> - add static route to original IP range 10.0.7.0/24
>
> - delete static route for 10.0.18.0/23
>
> - add back to cluster
>
> For each OSD:
>
> - stop OSD daemons
>
> - change IP address
>
> - add static route to original IP range 10.0.7.0/24
>
> - delete static route for 10.0.18.0/23
>
> - start OSD daemons
>
> c. Clean up all static routes defined.
>
>
>
> 2.
>
> a. Export and update monmap using the messy way as described in
> http://docs.ceph.com/docs/mimic/rados/operations/add-or-rm-mons/
>
>
>
> ceph mon getmap -o {tmp}/{filename}
>
> monmaptool -rm node1 -rm node2 ... --rm node n {tmp}/{filename}
>
> monmaptool -add node1 v2:10.0.18.1:3330,v1:10.0.18.1:6789 -add node2
> v2:10.0.18.2:3330,v1:10.0.18.2:6789 ... --add nodeN
> v2:10.0.18.N:3330,v1:10.0.18.N:6789 {tmp}/{filename}
>
>
>
> b. stop entire cluster daemons and change IP addresses
>
>
> c. For each mon node: ceph-mon -I {mon-id} -inject-monmap
> {tmp}/{filename}
>
>
>
> d. Restart cluster daemons.
>
>
>
> 3. Or any better method...
> =====================
>
> Would anyone please help? Thanks a lot.
> Rgds
> /st wong
>
--
Manuel Lausch
Systemadministrator
Storage Services
1&1 Mail & Media Development & Technology GmbH | Brauerstraße 48 |
76135 Karlsruhe | Germany Phone: +49 721 91374-1847
E-Mail: manuel.lausch@xxxxxxxx | Web: www.1und1.de
Hauptsitz Montabaur, Amtsgericht Montabaur, HRB 5452
Geschäftsführer: Alexander Charles, Thomas Ludwig, Jan Oetjen, Sascha Vollmer
Member of United Internet
Diese E-Mail kann vertrauliche und/oder gesetzlich geschützte Informationen enthalten. Wenn Sie nicht der bestimmungsgemäße Adressat sind oder diese E-Mail irrtümlich erhalten haben, unterrichten Sie bitte den Absender und vernichten Sie diese E-Mail. Anderen als dem bestimmungsgemäßen Adressaten ist untersagt, diese E-Mail zu speichern, weiterzuleiten oder ihren Inhalt auf welche Weise auch immer zu verwenden.
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient of this e-mail, you are hereby notified that saving, distribution or use of the content of this e-mail in any way is prohibited. If you have received this e-mail in error, please notify the sender and delete the e-mail.
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
