On 2019-06-07 08:36, Ilya Maximets wrote:
On 06.06.2019 21:03, Jonathan Lemon wrote:On 6 Jun 2019, at 5:40, Ilya Maximets wrote:Device that bound to XDP socket will not have zero refcount until the userspace application will not close it. This leads to hang inside 'netdev_wait_allrefs()' if device unregistering requested: # ip link del p1 < hang on recvmsg on netlink socket > # ps -x | grep ip 5126 pts/0 D+ 0:00 ip link del p1 # journalctl -b Jun 05 07:19:16 kernel: unregister_netdevice: waiting for p1 to become free. Usage count = 1 Jun 05 07:19:27 kernel: unregister_netdevice: waiting for p1 to become free. Usage count = 1 ... Fix that by counting XDP references for the device and failing RTM_DELLINK with EBUSY if device is still in use by any XDP socket. With this change: # ip link del p1 RTNETLINK answers: Device or resource busy Fixes: 965a99098443 ("xsk: add support for bind for Rx") Signed-off-by: Ilya Maximets <i.maximets@xxxxxxxxxxx> --- Another option could be to force closing all the corresponding AF_XDP sockets, but I didn't figure out how to do this properly yet. include/linux/netdevice.h | 25 +++++++++++++++++++++++++ net/core/dev.c | 10 ++++++++++ net/core/rtnetlink.c | 6 ++++++ net/xdp/xsk.c | 7 ++++++- 4 files changed, 47 insertions(+), 1 deletion(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 44b47e9df94a..24451cfc5590 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -1705,6 +1705,7 @@ enum netdev_priv_flags { * @watchdog_timer: List of timers * * @pcpu_refcnt: Number of references to this device + * @pcpu_xdp_refcnt: Number of XDP socket references to this device * @todo_list: Delayed register/unregister * @link_watch_list: XXX: need comments on this one * @@ -1966,6 +1967,7 @@ struct net_device { struct timer_list watchdog_timer; int __percpu *pcpu_refcnt; + int __percpu *pcpu_xdp_refcnt; struct list_head todo_list;I understand the intention here, but don't think that putting a XDP reference into the generic netdev structure is the right way of doing this. Likely the NETDEV_UNREGISTER notifier should be used so the socket and umem unbinds from the device.Thanks for the pointer! That is exactly what I looked for. I'll make a new version that will unbind resources using netdevice notifier. Best regards, Ilya Maximets.
Thanks for working on this. This would open up for supporting killing sockets via ss(8) (-K, --kill), as well! Björn
|