Spurious device addition uevents sometimes get sent after removing a loop device

[Ondřej Lysoněk <olysonek@xxxxxxxxxx>]

Hi,

I'd like to report a problem I ran into, where the kernel sends device
removal uevents immediately followed by spurious device addition
uevents. This happens after removing a loop device using the
LOOP_CTL_REMOVE ioctl on /dev/loop-control.

I can reproduce it with the following program on a vanilla 5.0-rc4
kernel, as well as Fedora's kernel-4.20.4-200.fc29.x86_64.

#include <err.h>
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <sys/ioctl.h>
#include <linux/loop.h>
#include <unistd.h>
#include <errno.h>

int main(void)
{
	const int devnum = 0;
	int ctl_fd;
	int removed = 0;

	ctl_fd = open("/dev/loop-control", O_RDWR);
	if (ctl_fd == -1)
		err(1, "ctl open");
	if (ioctl(ctl_fd, LOOP_CTL_ADD, devnum) == -1)
		err(1, "add ioctl");

	while (!removed) {
		if (ioctl(ctl_fd, LOOP_CTL_REMOVE, devnum) == -1) {
			/* Sometimes we get EBUSY here.
			 * Is that expected? */
			if (errno != EBUSY) {
				err(1, "ioctl");
			}
		} else {
			removed = 1;
		}
	}
	if (close(ctl_fd) == -1) {
		err(1, "close");
	}

	return 0;
}

If I run it like this:
$ while sudo ./reproducer && ! test -e /dev/loop0; do true; done

then after about a minute the loop terminates without any errors and
/dev/loop0 exists.

Monitoring the uevents sent by the kernel (udevadm monitor --kernel)
reveals that during most of the executions of the program, the following
uevents get sent:
KERNEL[99.624241] add      /devices/virtual/bdi/7:0 (bdi)
KERNEL[99.624392] add      /devices/virtual/block/loop0 (block)
KERNEL[99.624779] remove   /devices/virtual/bdi/7:0 (bdi)
KERNEL[99.624845] remove   /devices/virtual/block/loop0 (block)

However, sometimes the removal uevents are followed by spurious device
addition uevents:
KERNEL[153.723097] add      /devices/virtual/bdi/7:0 (bdi)
KERNEL[153.723172] add      /devices/virtual/block/loop0 (block)
KERNEL[153.723406] remove   /devices/virtual/bdi/7:0 (bdi)
KERNEL[153.724265] remove   /devices/virtual/block/loop0 (block)
KERNEL[153.746690] add      /devices/virtual/bdi/7:0 (bdi)
KERNEL[153.746931] add      /devices/virtual/block/loop0 (block)

This results in /dev/loop0 being readded by udev.

I can reproduce the problem more easily if I actually bind an image file
to the loop device, e.g. like this (the referenced file.img is just
filled with zeros):
https://paste.fedoraproject.org/paste/o6jqJV8pBiTEk8lXIMcX-g

Have you seen this?

Thanks!

Best regards
Ondřej Lysoněk