Error creating a AF_XDP socket after deleting

[Jesse DuMond <suttonshiredev@xxxxxxxxx>]

Hi there,

I am writing some test code that tries  repeatedly to create a umem
and socket and then deletes them both. I'm using the libbpf function
xsk_umem__create/delete and xsk_socket__create/xsk_socket__delete. I
am getting an EBUSY from xsk_socket__create when trying to create a
socket after deleting a socket on the same interface and queue number.
The attached program reproduces the error.

I've traced the EBUSY error to this call to xdp_umem_assign_dev in the
kernel: https://github.com/torvalds/linux/blob/v5.3/net/xdp/xsk.c#L500
That call is made during the bind call in xsk_socket__create. As far
as I can tell this error is because the work of xdp_umem_clear_dev is
actually called asynchronously after xsk_socket__delete. Putting a
sleep between xsk_socket__delete and xsk_socket__create makes this
code succeed.

Am I trying to do something that is not supposed to be supported?

/* SPDX-License-Identifier: GPL-2.0 */
#include <errno.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <sys/mman.h>
#include <unistd.h>

#include <bpf/libbpf.h>
#include <bpf/xsk.h>
#include <bpf/bpf.h>

int main(int argc, char **argv)
{
	void *bufs;
	struct xsk_umem *umem;
	struct xsk_socket *socket;
	struct xsk_ring_prod fq;
	struct xsk_ring_cons cq;
	struct xsk_ring_prod tx;
	struct xsk_ring_cons rx;
	struct xsk_umem_config ucfg = {
		.fill_size = 4,
		.comp_size = 4,
		.frame_size = 2048,
		.frame_headroom = 0,
		.flags = 0
	};
	struct xsk_socket_config scfg = {
		.rx_size = 4,
		.tx_size = 4,
		.libbpf_flags = XSK_LIBBPF_FLAGS__INHIBIT_PROG_LOAD,
		.bind_flags = 0,
		.xdp_flags = 0
	};
	int ret;

	bufs = mmap(NULL, 4 * 2048,
		    PROT_READ | PROT_WRITE,
		    MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
	if (bufs == MAP_FAILED) {
		printf("ERROR: mmap failed\n");
		exit(EXIT_FAILURE);
	}

	ret = xsk_umem__create(&umem, bufs, 4*2048, &fq, &cq, &ucfg);
	if (ret) {
		printf("ERROR: xsk_umem__create failed: %d (%s)\n", -ret, strerror(-ret));
		exit(1);
	}


	ret = xsk_socket__create(&socket, "test", 0, umem, &rx, &tx, &scfg);
	if (ret) {
		printf("ERROR: xsk_socket__create failed: %d (%s)\n", ret, strerror(-ret));
		exit(1);
	}

	xsk_socket__delete(socket);
	xsk_umem__delete(umem);
	//sleep(1);
	ret = xsk_umem__create(&umem, bufs, 4*2048, &fq, &cq, &ucfg);
	if (ret) {
		printf("ERROR: second xsk_umem__create failed: %d (%s)\n", -ret, strerror(-ret));
		exit(1);
	}


	ret = xsk_socket__create(&socket, "test", 0, umem, &rx, &tx, &scfg);
	if (ret) {
		printf("ERROR: second xsk_socket__create failed: %d (%s)\n", ret, strerror(-ret));
		exit(1);
	}

	return 0;
}