Re: Glusterfs fuse uses system ports to mount remote bricks

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Wed, Jun 13, 2018 at 6:12 PM, Canh Ngo <canhnt@xxxxxxxxx> wrote:
Hi all,

We run a storage cluster using GlusterFS v3.10.12 on CentOS7. Clients (CentOS) are using glusterfs 3.8.4.

We notice when clients mounts bricks of a volume, sometimes glusterfs uses system ports (i.e. in port range 0-1024) to connect to remote glusterfsd port. e.g:

Server:
tcp        0      0 10.165.210.169:49161    10.165.210.51:850       ESTABLISHED 32760/glusterfsd

Client:
tcp        0      0 10.165.210.51:850       10.165.210.169:49161    ESTABLISHED 25483/glusterfs

Thus, sometimes glusterfs occupies our system ports (e.g. 995, 179, 443, etc), that causes other services cannot start.

Is is a bug or an expected behavior? I expect glusterfs should use IANA private ports rather than system ports. Do you know if we can configure glusterfs to use a specific port range?

​Thanks,
Canh Ngo.

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://lists.gluster.org/mailman/listinfo/gluster-users



Here's some info on the kernel (sysctl) tunables that you could tweak:
/proc/sys/net/ipv4/ip_local_port_range
/proc/sys/net/ipv4/ip_local_reserved_ports

Here's what networking/ip-sysctl.txt from the kernel documentation directory says:
ip_local_port_range - 2 INTEGERS
        Defines the local port range that is used by TCP and UDP to
        choose the local port. The first number is the first, the
        second the last local port number.
        If possible, it is better these numbers have different parity.
        (one even and one odd values)
        The default values are 32768 and 60999 respectively.

ip_local_reserved_ports - list of comma separated ranges
        Specify the ports which are reserved for known third-party
        applications. These ports will not be used by automatic port
        assignments (e.g. when calling connect() or bind() with port
        number 0). Explicit port allocation behavior is unchanged.

        The format used for both input and output is a comma separated
        list of ranges (e.g. "1,2-4,10-10" for ports 1, 2, 3, 4 and
        10). Writing to the file will clear all previously reserved
        ports and update the current list with the one given in the
        input.

        Note that ip_local_port_range and ip_local_reserved_ports
        settings are independent and both are considered by the kernel
        when determining which ports are available for automatic port
        assignments.

        You can reserve ports which are not in the current
        ip_local_port_range, e.g.:

        $ cat /proc/sys/net/ipv4/ip_local_port_range
        32000   60999
        $ cat /proc/sys/net/ipv4/ip_local_reserved_ports
        8080,9148

        although this is redundant. However such a setting is useful
        if later the port range is changed to a value that will
        include the reserved ports.

        Default: Empty

You could check the values of these files on your system and configure them accordingly. Gluster specifically looks at /proc/sys/net/ipv4/ip_local_reserved_ports to avoid assigning values from the reserved port range. Alternatively, you could configure the system via /etc/sysctl.conf to persist the settings across reboots:

net.ipv4.ip_local_reserved_ports
net.ipv4.ip_local_port_range

Hope this helps.

--
Milind
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://lists.gluster.org/mailman/listinfo/gluster-users

[Index of Archives]     [Gluster Development]     [Linux Filesytems Development]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux