systemd-networkd packet loss Ubuntu 18.04

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,

I had a look through some of the issues against the systemd github repo but could not find any matching - there were a couple which look promising but don't completely resolve our issue.

Our issue is that we have a k8s (1.18, kube-router CNI) cluster comprised of a number of Ubuntu 16.04 nodes, we are in the process if upgrading the nodes to Ubuntu 18.04 - however after upgrading the first node and adding it back to the cluster we observed high packet loss from other cluster nodes to the 18.04 node as well as ping error messages when running a continuous ping from the 18.04 node to another node in the cluster, eg:

64 bytes from 10.8.11.1: icmp_seq=91 ttl=64 time=0.088 ms 
64 bytes from 10.8.11.1: icmp_seq=92 ttl=64 time=0.076 ms 
ping: sendmsg: No buffer space available 
ping: sendmsg: No buffer space available 
ping: sendmsg: No buffer space available

Another observation is that one of our daemonset pods (promtail) could not start.

systemctl --version
systemd 237
+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN2 +IDN -PCRE2 default-hierarchy=hybrid

After many days of troubleshooting various possibilities, changing network cable, NIC, a different server, various sysctl values, I eventually tried switching from systemd-networkd to NetworkManager as the backend via netplan config. After rebooting the server with NetworkManager in place and re-adding the node back to the k8s cluster, the packet loss disappeared as did the ping errors and the daemonset pod (promtail) also started normally.

I would like to use Ubuntu's default of systemd-networkd to handle our NIC/route config - but cannot until I've figured out what in systemd-networkd is breaking.

Any help is much appreciated on this matter.

Thanks!
Luke


_______________________________________________
systemd-devel mailing list
systemd-devel@xxxxxxxxxxxxxxxxxxxxx
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

[Index of Archives]     [LARTC]     [Bugtraq]     [Yosemite Forum]     [Photo]

  Powered by Linux