Re: Unable to change port range for OSDs in Pacific

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Good morning Eugen!


Thank you, this allowed me to succcessfully migrate my OSDs to ports above 6830. This in turn prevents the conflict with slurmd.


Cordially,

Renata.

On 5/18/23 18:26, Eugen Block wrote:
Hi,
the config options you mention should work, but not in the ceph.conf. You should set it via ‚ceph config set …‘ and then restart the daemons (ceph orch daemon restart osd).

Zitat von Renata Callado Borges <renato.callado@xxxxxxxxxxxx>:

Dear all,


How are you?

I have a Pacific 3 nodes cluster, and the machines do double-duty as Ceph nodes and as Slurm clients. (I am well aware that this is not desirable, but my client wants it like this anyway).

Our Slurm install uses the port 6818 for slurmd everywhere.

In one of our Ceph/Slurm nodes, Ceph decided that port 6818 is great for an OSD. This prevents slurmd from running properly. Changing the slurmd port causes the Slurm master, slurmctld, to misread the OSD communication as Slurm "Insane length messages".

I have tried unsuccessfully to change this port in Slurm and in Ceph. I wonder if someone here can help me limit the ports Ceph uses for its OSDs.

I have tried this with no success:

[root@darkside1 ~]# cat /etc/ceph/ceph.conf~
# minimal ceph.conf for 1902a026-496d-11ed-b43e-08c0eb320ec2
[global]
        fsid = 1902a026-496d-11ed-b43e-08c0eb320ec2
        mon_host = [v2:172.22.132.188:3300/0,v1:172.22.132.188:6789/0]

[osd]
        ms_bind_port_min = 6830
        ms_bind_port_max = 7300


Then I restarted with "systemctl restart ceph.target" but the OSD keeps being re-bound to 6818. I also tried the same config but with the options under the [global] section. No luck there either. Tried reboot the Ceph/Slurm machine, the OSD is re-bound in 6818 also.

Could someone help? Thanks in advance!


Cordially,

Renata.
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx


_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux