Re: Error creating a AF_XDP socket after deleting

[Jesse DuMond <suttonshiredev@xxxxxxxxx>]

Hmm in the attached example the first retry after an EBUSY returns EINVAL.

On Mon, Apr 27, 2020 at 2:53 AM Maciej Fijalkowski
<maciej.fijalkowski@xxxxxxxxx> wrote:
>
> On Sun, Apr 26, 2020 at 08:38:03PM +0200, Magnus Karlsson wrote:
> > On Sun, Apr 26, 2020 at 6:08 PM Jesse DuMond <suttonshiredev@xxxxxxxxx> wrote:
> > >
> > > Hi there,
> > >
> > > I am writing some test code that tries  repeatedly to create a umem
> > > and socket and then deletes them both. I'm using the libbpf function
> > > xsk_umem__create/delete and xsk_socket__create/xsk_socket__delete. I
> > > am getting an EBUSY from xsk_socket__create when trying to create a
> > > socket after deleting a socket on the same interface and queue number.
> > > The attached program reproduces the error.
> > >
> > > I've traced the EBUSY error to this call to xdp_umem_assign_dev in the
> > > kernel: https://github.com/torvalds/linux/blob/v5.3/net/xdp/xsk.c#L500
> > > That call is made during the bind call in xsk_socket__create. As far
> > > as I can tell this error is because the work of xdp_umem_clear_dev is
> > > actually called asynchronously after xsk_socket__delete. Putting a
> > > sleep between xsk_socket__delete and xsk_socket__create makes this
> > > code succeed.
> > >
> > > Am I trying to do something that is not supposed to be supported?
> >
> > No, you are doing everything correctly. The tear down of a socket is
> > unfortunately asynchronous due to legacy reasons that are not true
> > anymore. We do have a item in the backlog to make this synchronous,
> > but have not gotten around to it yet. So sorry, we have to live with
> > this for a while more.
>
> For now I guess that in case of EBUSY the application can just repeat the
> call to xsk_socket__create?
>
> >
> > /Magnus

#include <errno.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <sys/mman.h>
#include <unistd.h>

#include <bpf/libbpf.h>
#include <bpf/xsk.h>
#include <bpf/bpf.h>

int main(int argc, char **argv)
{
	void *bufs;
	struct xsk_umem *umem;
	struct xsk_socket *socket;
	struct xsk_ring_prod fq;
	struct xsk_ring_cons cq;
	struct xsk_ring_prod tx;
	struct xsk_ring_cons rx;
	struct xsk_umem_config ucfg = {
		.fill_size = 4,
		.comp_size = 4,
		.frame_size = 2048,
		.frame_headroom = 0,
		.flags = 0
	};
	struct xsk_socket_config scfg = {
		.rx_size = 4,
		.tx_size = 4,
		.libbpf_flags = XSK_LIBBPF_FLAGS__INHIBIT_PROG_LOAD,
		.bind_flags = 0,
		.xdp_flags = 0
	};
	int ret;

	bufs = mmap(NULL, 4 * 2048,
		    PROT_READ | PROT_WRITE,
		    MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
	if (bufs == MAP_FAILED) {
		printf("ERROR: mmap failed\n");
		exit(EXIT_FAILURE);
	}

	ret = xsk_umem__create(&umem, bufs, 4*2048, &fq, &cq, &ucfg);
	if (ret) {
		printf("ERROR: xsk_umem__create failed: %d (%s)\n", -ret, strerror(-ret));
		exit(1);
	}


	ret = xsk_socket__create(&socket, "test", 0, umem, &rx, &tx, &scfg);
	if (ret) {
		printf("ERROR: xsk_socket__create failed: %d (%s)\n", ret, strerror(-ret));
		exit(1);
	}

	xsk_socket__delete(socket);
	xsk_umem__delete(umem);
	//sleep(1);
	ret = xsk_umem__create(&umem, bufs, 4*2048, &fq, &cq, &ucfg);
	if (ret) {
		printf("ERROR: second xsk_umem__create failed: %d (%s)\n", -ret, strerror(-ret));
		exit(1);
	}


	//This will fail
	ret = xsk_socket__create(&socket, "test", 0, umem, &rx, &tx, &scfg);
	if (ret == -EBUSY) {
		ret = xsk_socket__create(&socket, "test", 0, umem, &rx, &tx, &scfg);
		if (ret) {
			printf("ERROR: retry xsk_socket__create failed: %d (%s)\n", ret, strerror(-ret));
			exit(1);
		}
	}

	return 0;
}