Re: mdadm/Create wait_for_zero_forks is stuck

[Logan Gunthorpe <logang@xxxxxxxxxxxx>]

Hi Xaio,

Sorry it took so long but I had a chance to dig into the bug today. It's
not what I had originally thought, but I do have a solution.

Turns out the problem is that multiple SIGCHLD signals can be coalesced
into one signal if they happen at the same time between reads to the
signalfd. This is just the way Linux works and I didn't account for it
in the code.

To fix this we, need to wait for multiple potential children being
completed after every SIGCHLD is received.

I've made two patches which you can get from:

https://github.com/lsgunth/mdadm/commits/write_zeros_sigbug/

I tested it with several hundred runs of your test script and it seems
to fix the problem. Please review and test for yourself.

On 2024-05-22 20:05, Xiao Ni wrote:
> I did a test in a simple c program.

I made a similar test program to try it out and I think the reason it
wasn't working for you was due to the coalescing and simply blocking
solves the (now only theoretical) race at startup. Once the coalescing
problem is fixed we still need to move the block earlier to fix the
race. I've attached the code for that program if you want to try it out.

Thanks for finding and triaging the bug!

Logan


#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/signalfd.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <unistd.h>


static pid_t do_fork(int sleep_for)
{
	pid_t pid;

	pid = fork();
	if (pid != 0)
		return pid;

	sleep(sleep_for);
	printf("done: %d\n", getpid());
	exit(0);
}

int main()
{
	struct signalfd_siginfo fdsi;
	int i, sfd, wst, ret = 0;
	sigset_t sigset;
	pid_t pid;
	ssize_t s;

	sigemptyset(&sigset);
	sigaddset(&sigset, SIGINT);
	sigaddset(&sigset, SIGCHLD);
	sigprocmask(SIG_BLOCK, &sigset, NULL);

	for (i = 0; i < 5; i++) {
		pid = do_fork(i);
		if (pid < 0) {
			fprintf(stderr, "Error forking %d: %m\n", i);
			ret = 1;
			break;
		}
	}

	sleep(2);

	sfd = signalfd(-1, &sigset, 0);
	if (sfd < 0) {
		fprintf(stderr, "Unable to create signalfd: %m\n");
		return 1;
	}

	while (i) {
		s = read(sfd, &fdsi, sizeof(fdsi));
		if (s != sizeof(fdsi)) {
			fprintf(stderr, "Invalid signalfd read: %m\n");
			return 1;
		}

		if (fdsi.ssi_signo == SIGINT) {
			fprintf(stderr, "Interrupted\n");
			ret = 1;
		} else if (fdsi.ssi_signo == SIGCHLD) {
			printf("sigchld\n");

			while (1) {
				pid = waitpid(-1, &wst, WNOHANG);
				if (pid <= 0)
					break;
				printf("waited: %d\n", pid);
				i--;
			}
		}
	}

	close(sfd);

	return ret;
}