On Mon, 29 Apr 2019 at 22:20, William Tu <u9012063@xxxxxxxxx> wrote:
>
> Hi,
>
> I'm adding AF_XDP support to OVS but hit a problem when users
> reconfigure the AF_XDP. Ex: users can use SKB mode in the beginning
> and then switch to DRV mode. So I want to cleanup the XSK and
> re-initialize everything. However it always fails.
>
> I simplifies the issue using xdpsock_user.c below:
> Assume user want to delete the previous xsk and recreate a new one.
> Now it fails at bind() returning "Device or resource busy"
>
> diff --git a/samples/bpf/xdpsock_user.c b/samples/bpf/xdpsock_user.c
> index d08ee1ab7bb4..f3e37f0451b5 100644
> --- a/samples/bpf/xdpsock_user.c
> +++ b/samples/bpf/xdpsock_user.c
> @@ -644,6 +644,17 @@ static void l2fwd(struct xsk_socket_info *xsk)
> }
> }
>
> +void cleanup(void)
> +{
> + struct xsk_umem *umem = xsks[0]->umem->umem;
> + printf("cleanup\n");
> +
> + xsk_socket__delete(xsks[0]->xsk);
> + (void)xsk_umem__delete(umem);
> + remove_xdp_program();
> + printf("cleanup done\n");
> +}
> +
> int main(int argc, char **argv)
> {
> struct rlimit r = {RLIM_INFINITY, RLIM_INFINITY};
> @@ -670,6 +681,23 @@ int main(int argc, char **argv)
> NUM_FRAMES * XSK_UMEM__DEFAULT_FRAME_SIZE);
> xsks[num_socks++] = xsk_configure_socket(umem);
>
> + cleanup();
> + num_socks = 0;
> + umem = NULL;
> +
> + printf("create again\n");
> + ret = posix_memalign(&bufs, getpagesize(), /* PAGE_SIZE aligned */
> + NUM_FRAMES * XSK_UMEM__DEFAULT_FRAME_SIZE);
> + if (ret)
> + exit_with_error(ret);
> +
> + /* Create sockets... */
> + umem = xsk_configure_umem(bufs,
> + NUM_FRAMES * XSK_UMEM__DEFAULT_FRAME_SIZE);
> + xsks[num_socks++] = xsk_configure_socket(umem);
> +
> + printf("create again done\n");
> +
> if (opt_bench == BENCH_TXONLY) {
> int i;
> ----
> // How to reproduce
> ip link add p0 type veth peer name afxdp-p0
> ip link set dev afxdp-p0 up
> ./xdpsock -S -r -i afxdp-p0
> /root/bpf-next/samples/bpf/xdpsock_user.c:xsk_configure_socket:315: errno: 16/"Device or resource busy"
>
> I think at net/xdp/xsk.c the xsk_release does not get invoke, so the
> xs->dev is not NULL.
>
> Still debugging and any suggestions are welcome.
>
> btw: tested on bpf-next commit a21b48a2f2afa53bbc989c
>
> Regards,
> William
Nice catch William!
You triggered two bugs in libbpf; munmap was broken, and the map
descriptors were leaking, leading to incorrect cleanup.
I'll send out fixes for that ASAP.
Björn
|