Interestingly enough this happens for some pools and not for others.
For example I have just realized that when trying to connect to another
pool the client is correctly directed to the OSD public_network address:
>> strace -f -e trace=network -s 10000 rbd ls --pool cinder-volumes --name
client.cinder 2>&1| grep sin_addr
[pid 2363212] connect(15, {sa_family=AF_INET, sin_port=htons(6816),
sin_addr=inet_addr("*10.113.29.7*")}, 16) = 0
But same client listing the ephemeral-vms pools is directed to the OSD
cluster address:
>> strace -f -e trace=network -s 10000 rbd ls --pool ephemeral-vms --name
client.cinder 2>&1| grep sin_addr
[pid 2363485] connect(14, {sa_family=AF_INET, sin_port=htons(6806),
sin_addr=inet_addr("*10.114.29.10*")}, 16) = -1 EINPROGRESS (Operation now
in progress)
Very weird!
On Tue, 28 Sept 2021 at 16:02, Javier Cacheiro <Javier.Cacheiro@xxxxxxxxx>
wrote:
> Hi all,
>
> I am trying to understand a issue with ceph directing clients to connect
> to OSDs through their cluster_network address instead of their
> public_network address.
>
> I have a configured a ceph cluster with a public and cluster network:
>
> >> ceph config dump|grep network
> global advanced cluster_network *10.114.0.0/16
> <http://10.114.0.0/16>* *
> mon advanced public_network 10.113.0.0/16 *
>
> I upgraded the cluster from 16.2.4 to 16.2.6.
>
> After that, I am seeing that ceph is directing clients to connect to OSD's
> cluster_network address instead of their public_address:
>
> >> strace -f -e trace=network -s 10000 rbd ls --pool ephemeral-vms --name
> client.cinder
> ....
> [pid 2353692] connect(14, {sa_family=AF_INET, sin_port=htons(6806),
> sin_addr=inet_addr("*10.114.29.10*")}, 16) = -1 EINPROGRESS (Operation
> now in progress)
>
> In this case the client hangs because it is not able to access the
> address, since its an internal address.
>
> This appeared after upgrading to 16.2.6, but I am not sure it was due to
> the upgrade or it was a hidden issue that appeared after the nodes were
> rebooted.
>
> It can also be that I am missing something in the config, but this config
> was generated by the cephadm bootstrap command and not created by hand, and
> it worked before the upgrade/reboot so I am pretty confident with it.
>
> What do you think, can this be a bug or is more a misconfiguration on my
> side?
>
> Thanks,
> Javier
>
>
>
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
