We're working on the migration to cephadm, but it requires some
prerequisites that still needs planing.
root@host:~# cat /etc/ceph/ceph.conf ; ceph config dump
[global]
fsid = ...
mon_host = [OLD_NETWORK::10], [OLD_NETWORK::11], [OLD_NETWORK::12]
#public_network = OLD_NETWORK::/64, NEW_NETWORK::/64
ms_bind_ipv6 = true
ms_bind_ipv4 = false
auth_cluster_required = none
auth_service_required = none
auth_client_required = none
[client....]
ms_mon_client_mode = crc
#debug_rgw = 20
rgw_frontends = beast endpoint=[OLD_NETWORK::12]:7480
rgw_region = ...
rgw_zone = ...
rgw_thread_pool_size = 512
rgw_dns_name = ...
rgw_dns_s3website_name = ...
[mon....-new]
public_addr = fNEW_NETWORK::12
public_bind_addr = NEW_NETWORK::12
WHO MASK LEVEL OPTION
VALUE
RO
global advanced auth_client_required
none
*
global advanced auth_cluster_required
none
*
global advanced auth_service_required
none
*
global advanced mon_allow_pool_size_one
true
global advanced ms_bind_ipv4
false
global advanced ms_bind_ipv6
true
global advanced osd_pool_default_pg_autoscale_mode
warn
global advanced public_network
OLD_NETWORK::/64, NEW_NETWORK::/64
*
mon advanced auth_allow_insecure_global_id_reclaim
false
mon advanced mon_allow_pool_delete
false
mgr advanced mgr/balancer/active
true
mgr advanced mgr/balancer/mode
upmap
mgr advanced mgr/cephadm/migration_current 5
*
mgr advanced mgr/orchestrator/orchestrator
cephadm
mgr.0cc47a6df14e basic container_image
quay.io/ceph/ceph@sha256:09e527353463993f0441ad3e86be98076c89c34552163e558a8c2f9bfb4a35f5
*
mgr.0cc47aad8ce8 basic container_image
quay.io/ceph/ceph@sha256:09e527353463993f0441ad3e86be98076c89c34552163e558a8c2f9bfb4a35f5
*
osd.0 basic osd_mclock_max_capacity_iops_ssd
13295.404086
osd.1 basic osd_mclock_max_capacity_iops_ssd
14952.522452
osd.2 basic osd_mclock_max_capacity_iops_ssd
13584.113025
osd.3 basic osd_mclock_max_capacity_iops_ssd
16421.770356
osd.4 basic osd_mclock_max_capacity_iops_ssd
15209.375302
osd.5 basic osd_mclock_max_capacity_iops_ssd
15333.697366
Am Mo., 21. Aug. 2023 um 14:20 Uhr schrieb Eugen Block <eblock@xxxxxx>:
> Hi,
>
> > I don't have those configs. The cluster is not maintained via cephadm /
> > orchestrator.
>
> I just assumed that with Quincy it already would be managed by
> cephadm. So what does the ceph.conf currently look like on an OSD host
> (mask sensitive data)?
>
> Zitat von Boris Behrens <bb@xxxxxxxxx>:
>
> > Hey Eugen,
> > I don't have those configs. The cluster is not maintained via cephadm /
> > orchestrator.
> > The ceph.conf does not have IPaddresses configured.
> > A grep in /var/lib/ceph show only binary matches on the mons
> >
> > I've restarted the whole host, which also did not work.
> >
> > Am Mo., 21. Aug. 2023 um 13:18 Uhr schrieb Eugen Block <eblock@xxxxxx>:
> >
> >> Hi,
> >>
> >> there have been a couple of threads wrt network change, simply
> >> restarting OSDs is not sufficient. I still haven't had to do it
> >> myself, but did you 'ceph orch reconfig osd' after adding the second
> >> public network, then restart them? I'm not sure if the orchestrator
> >> works as expected here, last year there was a thread [1] with the same
> >> intention. Can you check the local ceph.conf file
> >> (/var/lib/ceph/<FSID>/<SERVICE>/config) of the OSDs (or start with
> >> one) if it contains both public networks? I (still) expect the
> >> orchestrator to update that config as well. Maybe it's worth a bug
> >> report? If there's more to it than just updating the monmap I would
> >> like to see that added to the docs since moving monitors to a
> >> different network is already documented [2].
> >>
> >> Regards,
> >> Eugen
> >>
> >> [1] https://www.spinics.net/lists/ceph-users/msg75162.html
> >> [2]
> >>
> >>
> https://docs.ceph.com/en/quincy/cephadm/services/mon/#moving-monitors-to-a-different-network
> >>
> >> Zitat von Boris Behrens <bb@xxxxxxxxx>:
> >>
> >> > Hi,
> >> > I need to migrate a storage cluster to a new network.
> >> >
> >> > I added the new network to the ceph config via:
> >> > ceph config set global public_network "old_network/64, new_network/64"
> >> > I've added a set of new mon daemons with IP addresses in the new
> network
> >> > and they are added to the quorum and seem to work as expected.
> >> >
> >> > But when I restart the OSD daemons, the do not bind to the new
> >> addresses. I
> >> > would have expected that the OSDs try to bind to all networks but they
> >> are
> >> > only bound to the old_network.
> >> >
> >> > The idea was to add the new set of network config to the current
> storage
> >> > hosts, bind everything to ip addresses in both networks, shift over
> >> > workload, and then remove the old network.
> >> > _______________________________________________
> >> > ceph-users mailing list -- ceph-users@xxxxxxx
> >> > To unsubscribe send an email to ceph-users-leave@xxxxxxx
> >>
> >>
> >> _______________________________________________
> >> ceph-users mailing list -- ceph-users@xxxxxxx
> >> To unsubscribe send an email to ceph-users-leave@xxxxxxx
> >>
> >
> >
> > --
> > Die Selbsthilfegruppe "UTF-8-Probleme" trifft sich diesmal abweichend im
> > groüen Saal.
> > _______________________________________________
> > ceph-users mailing list -- ceph-users@xxxxxxx
> > To unsubscribe send an email to ceph-users-leave@xxxxxxx
>
>
> _______________________________________________
> ceph-users mailing list -- ceph-users@xxxxxxx
> To unsubscribe send an email to ceph-users-leave@xxxxxxx
>
--
Die Selbsthilfegruppe "UTF-8-Probleme" trifft sich diesmal abweichend im
groüen Saal.
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
