You'll need to update the mon_host line as well. Not sure if it makes
sense to have both old and new network in there, but I'd try on one
host first and see if it works.
Zitat von Boris Behrens <bb@xxxxxxxxx>:
We're working on the migration to cephadm, but it requires some
prerequisites that still needs planing.
root@host:~# cat /etc/ceph/ceph.conf ; ceph config dump
[global]
fsid = ...
mon_host = [OLD_NETWORK::10], [OLD_NETWORK::11], [OLD_NETWORK::12]
#public_network = OLD_NETWORK::/64, NEW_NETWORK::/64
ms_bind_ipv6 = true
ms_bind_ipv4 = false
auth_cluster_required = none
auth_service_required = none
auth_client_required = none
[client....]
ms_mon_client_mode = crc
#debug_rgw = 20
rgw_frontends = beast endpoint=[OLD_NETWORK::12]:7480
rgw_region = ...
rgw_zone = ...
rgw_thread_pool_size = 512
rgw_dns_name = ...
rgw_dns_s3website_name = ...
[mon....-new]
public_addr = fNEW_NETWORK::12
public_bind_addr = NEW_NETWORK::12
WHO MASK LEVEL OPTION
VALUE
RO
global advanced auth_client_required
none
*
global advanced auth_cluster_required
none
*
global advanced auth_service_required
none
*
global advanced mon_allow_pool_size_one
true
global advanced ms_bind_ipv4
false
global advanced ms_bind_ipv6
true
global advanced osd_pool_default_pg_autoscale_mode
warn
global advanced public_network
OLD_NETWORK::/64, NEW_NETWORK::/64
*
mon advanced auth_allow_insecure_global_id_reclaim
false
mon advanced mon_allow_pool_delete
false
mgr advanced mgr/balancer/active
true
mgr advanced mgr/balancer/mode
upmap
mgr advanced mgr/cephadm/migration_current 5
*
mgr advanced mgr/orchestrator/orchestrator
cephadm
mgr.0cc47a6df14e basic container_image
quay.io/ceph/ceph@sha256:09e527353463993f0441ad3e86be98076c89c34552163e558a8c2f9bfb4a35f5
*
mgr.0cc47aad8ce8 basic container_image
quay.io/ceph/ceph@sha256:09e527353463993f0441ad3e86be98076c89c34552163e558a8c2f9bfb4a35f5
*
osd.0 basic osd_mclock_max_capacity_iops_ssd
13295.404086
osd.1 basic osd_mclock_max_capacity_iops_ssd
14952.522452
osd.2 basic osd_mclock_max_capacity_iops_ssd
13584.113025
osd.3 basic osd_mclock_max_capacity_iops_ssd
16421.770356
osd.4 basic osd_mclock_max_capacity_iops_ssd
15209.375302
osd.5 basic osd_mclock_max_capacity_iops_ssd
15333.697366
Am Mo., 21. Aug. 2023 um 14:20 Uhr schrieb Eugen Block <eblock@xxxxxx>:
Hi,
> I don't have those configs. The cluster is not maintained via cephadm /
> orchestrator.
I just assumed that with Quincy it already would be managed by
cephadm. So what does the ceph.conf currently look like on an OSD host
(mask sensitive data)?
Zitat von Boris Behrens <bb@xxxxxxxxx>:
> Hey Eugen,
> I don't have those configs. The cluster is not maintained via cephadm /
> orchestrator.
> The ceph.conf does not have IPaddresses configured.
> A grep in /var/lib/ceph show only binary matches on the mons
>
> I've restarted the whole host, which also did not work.
>
> Am Mo., 21. Aug. 2023 um 13:18 Uhr schrieb Eugen Block <eblock@xxxxxx>:
>
>> Hi,
>>
>> there have been a couple of threads wrt network change, simply
>> restarting OSDs is not sufficient. I still haven't had to do it
>> myself, but did you 'ceph orch reconfig osd' after adding the second
>> public network, then restart them? I'm not sure if the orchestrator
>> works as expected here, last year there was a thread [1] with the same
>> intention. Can you check the local ceph.conf file
>> (/var/lib/ceph/<FSID>/<SERVICE>/config) of the OSDs (or start with
>> one) if it contains both public networks? I (still) expect the
>> orchestrator to update that config as well. Maybe it's worth a bug
>> report? If there's more to it than just updating the monmap I would
>> like to see that added to the docs since moving monitors to a
>> different network is already documented [2].
>>
>> Regards,
>> Eugen
>>
>> [1] https://www.spinics.net/lists/ceph-users/msg75162.html
>> [2]
>>
>>
https://docs.ceph.com/en/quincy/cephadm/services/mon/#moving-monitors-to-a-different-network
>>
>> Zitat von Boris Behrens <bb@xxxxxxxxx>:
>>
>> > Hi,
>> > I need to migrate a storage cluster to a new network.
>> >
>> > I added the new network to the ceph config via:
>> > ceph config set global public_network "old_network/64, new_network/64"
>> > I've added a set of new mon daemons with IP addresses in the new
network
>> > and they are added to the quorum and seem to work as expected.
>> >
>> > But when I restart the OSD daemons, the do not bind to the new
>> addresses. I
>> > would have expected that the OSDs try to bind to all networks but they
>> are
>> > only bound to the old_network.
>> >
>> > The idea was to add the new set of network config to the current
storage
>> > hosts, bind everything to ip addresses in both networks, shift over
>> > workload, and then remove the old network.
>> > _______________________________________________
>> > ceph-users mailing list -- ceph-users@xxxxxxx
>> > To unsubscribe send an email to ceph-users-leave@xxxxxxx
>>
>>
>> _______________________________________________
>> ceph-users mailing list -- ceph-users@xxxxxxx
>> To unsubscribe send an email to ceph-users-leave@xxxxxxx
>>
>
>
> --
> Die Selbsthilfegruppe "UTF-8-Probleme" trifft sich diesmal abweichend im
> groüen Saal.
> _______________________________________________
> ceph-users mailing list -- ceph-users@xxxxxxx
> To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
--
Die Selbsthilfegruppe "UTF-8-Probleme" trifft sich diesmal abweichend im
groüen Saal.
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
