Hello KJ
Unfortunately, I missed this message at the time...
On Sat, 18 Aug 2018 at 08:51, <ktsanaktsidis@xxxxxxxxxxx> wrote:
>
> From: KJ Tsanaktsidis <ktsanaktsidis@xxxxxxxxxxx>
>
> The clone(2) syscall (and consequently pthread_create(3)) can return
> the error ENOSPC when there are no more available PIDs on the system
> because the limit in /proc/sys/kernel/pid_max has been exceeded.
>
> The code path that can cause this error to be returned is:
> * kernel/fork.c do_fork
> * calls kernel/fork.c _do_fork
> * calls kernel/fork.c copy_process
> * calls kernel/pid.c alloc_pid
> * calls lib/idr.c idr_alloc_cyclic
> * calls lib/idr.h idr_alloc
> * calls lib/idr.c idr_alloc_cmn
> * calls include/linux/radix-tree.h idr_get_free
> * calls lib/radix-tree.c idr_get_free_cmn
> which looks like it can return ENOSPC when it can't grow the radix tree.
> The size of this tree looks like it's determined by pid_max (full
> disclaimer; I don't really understand any of the radix tree code).
>
> This behaviour can be reproduced by running this small test program on a
> system with a low value of /proc/sys/kernel/pid_max:
>
> #define _GNU_SOURCE
> #include <unistd.h>
> #include <sys/syscall.h>
> #include <stdio.h>
> #include <errno.h>
> #include <sched.h>
> #include <string.h>
>
> int main() {
> int n_children_created = 0;
> int fail_errno = 0;
> while (1) {
> int result = syscall(SYS_clone,
> CLONE_FILES | CLONE_FS | CLONE_IO | CLONE_SYSVSEM,
> 0, 0, 0, 0
> );
> if (result == 0) {
> return 0; // Child; just exit
> } else if (result == -1) {
> fail_errno = errno;
> break; // Failed; print our message.
> } else {
> n_children_created++;
> }
> }
> printf(
> "Got a clone(2) failure; %d (%s). Made %d children.\n",
> fail_errno, strerror(fail_errno), n_children_created
> );
> return 0;
> }
>
> $ cat /proc/sys/kernel/pid_max
> 30000
> $ gcc clonetest.c -o clonetest
> $ ./clonetest
> Got a clone(2) failure; 28 (No space left on device). Made 28822 children.
I *don't* reproduce this. On my Fedora 31 system, the error I see is
EAGAIN. Jakub says he reproduced what you see. I wonder if you both
run the same distro, and this is some distro-specific behavior. What
distro are you using?
Reading the kernel source (Linux 5.6), lib/idr.c::idr_alloc_cyclic()
does indeed return -ENOSPC when IDs are exhausted (AFAICT), but in
kernel/pid.c::alloc_pid(), there is this code:
nr = idr_alloc_cyclic(&tmp->idr, NULL, pid_min,
pid_max, GFP_ATOMIC);
}
spin_unlock_irq(&pidmap_lock);
idr_preload_end();
if (nr < 0) {
retval = (nr == -ENOSPC) ? -EAGAIN : nr;
goto out_free;
}
That's consistent with the behavior I see, and with what I expect.
Thanks,
Michael
> ---
> man2/clone.2 | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/man2/clone.2 b/man2/clone.2
> index e4bee67bf..262815b01 100644
> --- a/man2/clone.2
> +++ b/man2/clone.2
> @@ -1091,6 +1091,12 @@ Cannot allocate sufficient memory to allocate a task structure for the
> child, or to copy those parts of the caller's context that need to be
> copied.
> .TP
> +.B ENOSPC
> +the maximum number of PIDs,
> +.IR /proc/sys/kernel/pid_max ,
> +was reached; see
> +.BR proc (5).
> +.TP
> .BR ENOSPC " (since Linux 3.7)"
> .\" commit f2302505775fd13ba93f034206f1e2a587017929
> .B CLONE_NEWPID
> --
> 2.17.1
>
--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
