Hi,
I'd love to have more of a discussion on this. There are issues in the
code around IPv4 assumptions, and wider issues around identity that make
Gluster hard to operate on a dual-stack / many-network setup.
E.g., the assumption that a peer IP will have a hostname that resolves
to the same IP.
Paul
On Tue, 21 Sep 2021, Mohit Agrawal wrote:
Hi,
In gluster we do support one kind of address family (either IPV4 or IPV6)
and it depends on the user
what address family they want to use.It is a configurable option a user
can set the value in volfile . Here
It seems you are facing an issue as you mentioned " HPE is forcing IPV4
servers in to to an IPV6 address", i
think you can avoid an error if u pass address-family=inet6(
xlator-option="transport.address-family=inet6")
during mount a volume and in that case you should not face an issue.
For more please refer this https://github.com/gluster/glusterfs/pull/2666
Thanks,
Mohit Agrawal
On Tue, Sep 21, 2021 at 2:42 PM Erik Jacobson <erik.jacobson@xxxxxxx> wrote:
Dear devel team -
I botched the email address here. I type "hpcm-devel" like 30 times a
day so I mistyped that. Sorry about that.
Any advice appreciated and see attached patch that "gets it going for
us" but obviously not something you could release.
Erik
---------- Forwarded message ----------
From: Erik Jacobson <erik.jacobson@xxxxxxx>
To: gluster-users@xxxxxxxxxxx, hpcm-devel@xxxxxxxxxxx
Cc:
Bcc:
Date: Mon, 20 Sep 2021 16:46:12 -0500
Subject: [Gluster-users] gluster forcing IPV6 on our IPV4 servers,
glusterd fails (was gluster update question regarding new DNS resolution
requirement)
I pretended I'm a low-level C programmer with network and filesystem
experience for a few hours.
I'm not sure what the right solution is but what was happening was the
code was trying to treat our IPV4 hosts as AF_INET6 and the family was
incompatible with our IPV4 IP addresses. Yes, we need to move to IPV6
but we're hoping to do that on our own time (~50 years like everybody
else :)
I found a chunk of the code that seemed to be force-setting us to
AF_INET6.
While I'm sure it is not 100% the correct patch, the patch attached and
pasted below is working for me so I'll integrate it with our internal
build to continue testing.
Please let me know if there is a configuration item I missed or a
different way to do this. I added -devel to this email.
In the previous thread, you would have seen that we're testing a
hopeful change that will upgrade our deployed customers from gluster
7.9 to gluster 9.3.
Thank you!! Advice on next steps would be appreciated !!
diff -Narup glusterfs-9.3-ORIG/rpc/rpc-transport/socket/src/name.c
glusterfs-9.3-NEW/rpc/rpc-transport/socket/src/name.c
--- glusterfs-9.3-ORIG/rpc/rpc-transport/socket/src/name.c 2021-06-29
00:27:44.381408294 -0500
+++ glusterfs-9.3-NEW/rpc/rpc-transport/socket/src/name.c 2021-09-20
16:34:28.969425361 -0500
@@ -252,9 +252,16 @@ af_inet_client_get_remote_sockaddr(rpc_t
/* Need to update transport-address family if address-family is not
provided
to command-line arguments
*/
+ /* HPE This is forcing our IPV4 servers in to to an IPV6 address
+ * family that is not compatible with IPV4. For now we will just set
it
+ * to AF_INET.
+ */
+ /*
if (inet_pton(AF_INET6, remote_host, &serveraddr)) {
sockaddr->sa_family = AF_INET6;
}
+ */
+ sockaddr->sa_family = AF_INET;
/* TODO: gf_resolve is a blocking call. kick in some
non blocking dns techniques */
On Mon, Sep 20, 2021 at 11:35:35AM -0500, Erik Jacobson wrote:
I missed the other important log snip:
The message "E [MSGID: 101075] [common-utils.c:520:gf_resolve_ip6]
0-resolver: error in getaddrinfo [{family=10}, {ret=Address family for
hostname not supported}]" repeated 620 times between [2021-09-20
15:49:23.720633 +0000] and [2021-09-20 15:50:41.731542 +0000]
So I will dig in to the code some here.
On Mon, Sep 20, 2021 at 10:59:30AM -0500, Erik Jacobson wrote:
Hello all! I hope you are well.
We are starting a new software release cycle and I am trying to find a
way to upgrade customers from our build of gluster 7.9 to our build of
gluster 9.3
When we deploy gluster, we foribly remove all references to any host
names and use only IP addresses. This is because, if for any reason a
DNS server is unreachable, even if the peer files have IPs and DNS, it
causes glusterd to be unable to reach peers properly. We can't really
rely on /etc/hosts either because customers take artistic licene with
their /etc/hosts files and don't realize that problems that can cause.
So our deployed peer files look something like this:
uuid=46a4b506-029d-4750-acfb-894501a88977
state=3
hostname1=172.23.0.16
That is, with full intention, we avoid host names.
When we upgrade to gluster 9.3, we fall over with these errors and
gluster is now partitioned and the updated gluster servers can't reach
anybody:
[2021-09-20 15:50:41.731543 +0000] E
[name.c:265:af_inet_client_get_remote_sockaddr] 0-management: DNS
resolution failed on host 172.23.0.16
As you can see, we have defined on purpose everything using IPs but in
9.3 it appears this method fails. Are there any suggestions short of
putting real host names in peer files?
FYI
This supercomputer will be using gluster for part of its system
management. It is how we deploy the Image Objects (squashfs images)
hosted on NFS today and served by gluster leader nodes and also store
system logs, console logs, and other data.
https://www.olcf.ornl.gov/frontier/
Erik
________
Community Meeting Calendar:
Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://meet.google.com/cpu-eiue-hvk
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users
________
Community Meeting Calendar:
Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://meet.google.com/cpu-eiue-hvk
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users
________
Community Meeting Calendar:
Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://meet.google.com/cpu-eiue-hvk
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users
-------
Community Meeting Calendar:
Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://meet.google.com/cpu-eiue-hvk
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-devel
--
Paul Jakma | paul@xxxxxxxxx | @pjakma | Key ID: 0xD86BF79464A2FF6A
Fortune:
About the only thing on a farm that has an easy time is the dog.
-------
Community Meeting Calendar:
Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://meet.google.com/cpu-eiue-hvk
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-devel
