노시현 / 학생 / 전기·정보공학부 <sihyeonroh@xxxxxxxxx> writes: > It is an obvious linux kernel bug (wrong clearance of used data while error handling), and kernel drivers using this function can be affected. I wouldn't rule out the possiblity that this is a driver rather than kernel bug either. I will need to go look at the kernel code more closely but are you saying that after a driver calls a sequence of memremap_pages()/memunmap_pages() that any subsequent call to memremap_pages() will fail? Even if that were the case it shouldn't cause the below issue as the driver needs to deal with the failure. So I am having a bit of a hard time following some of the reasoning, comments below. > 1. Summary: Due to a bug in the Linux kernel, devices using the Linux kernel API cannot guarantee fault isolation between processes. > > 2. Full Description of the Problem > > (1) Overview of Problematic Functions > This section provides an overview of problematic functions, briefly explaining their purposes. Following three functions are responsible for handling the bug, broken device fault isolation. two are defined in <linux/mm/memremap.c>, and the other is defined in <linux/mm/sparse.c>. Function name, location, and brief explanation for understanding the problem are specified below. > > Function 1. <memremap_pages> > Source path: linux/mm/memremap.c > [cid:995e2ceb-43a8-4818-a023-9817801ac767] > Function 2. <section_activate> > (called by pagemap_range > add_pages > __add_pages > sparse_add_section > section_activate) > Source path: linux/mm/sparse.c > [cid:7ce32169-d894-452f-bc90-0036849ce79a] > Function 3. <memunmap_pages> > (called by memremap_pages > section_activate) > Source path: linux/mm/memremap.c > [cid:e90f753d-25b6-465f-b4f8-214ca254da00] > (2) Bug Triggering Flow > Let’s begin with assuming that process A calls memremap_pages with nr_range (the number of pages to allocate) 1. Nit: nr_range is the number of page ranges to allocate (ie. the number of struct range in the flex array), not the number of pages. Each range can contain multiple pages as controlled by range.start/end. > [cid:ec1e9d21-b358-4ff1-af38-da1a43ca7cd0] > > Above flow shows that if allocating memory in 864 line of section_activate function fails, the subsection_map masked by process A can never be cleared. This is because pageunmap_range is responsible for clearing subsection_map mask bit, but it can’t be called due to wrong nr_range count. > As the mask bit of subsection_map is not cleared, following call of memremap_pages from other processes ends up with failure, because given pfn is masked as busy by process A. > [cid:993c1924-9680-4981-8b99-0436b7e7a5c7]An error occurred in process A affects other processes using same pfn, which is usually the case of the processes that share the device with process A. The device driver using this linux kernel api can cause fatal vulnerability in security perspective. For example, NVIDIA guarantees GPU users a fault isolation between GPU-using processes. What makes the situation worse in CUDA programming is that checking for GPU errors is the user's responsibility. So, If users believe that GPU has a robust fault isolation, and uses it like TPM[1] or Security Engine Accelerator[2, 3], attacker can use this vulnerability to tear down GPU-based security systems. > (3) Bug usage by an attacker > Followings show how attackers can use this vulnerability, in security perspective. > [cid:c5b488fa-9ca1-4d44-99f8-847ca63d0387] > This is a classical parallel AES encryption implementation using CUDA, which tries to accelerate AES encryption through GPU. > Source code is from github repository, https://github.com/allenlee820202/Parallel-AES-Algorithm-using-CUDA. > This application encrypts strings, “Hello World!” written in novel.txt, using AES keys in key.txt. The encryption’s result is written into encrypt.txt, and its decryption is written into decrypt.txt. > [cid:b19783df-cf21-4659-9952-0d8ba6d18ad3] > [cid:a12df0c2-01e4-4564-8fee-05d9978abc9f] > You can see that encryption (“Hello world!” in novel.txt is encrypted into “d5 68 … “ in encrypt.txt) works well. However, in case this bug is triggered by another process using same GPU driver, the following shows GPU does not work, and encryption fails, resulting in plain text is stored in encrypt.txt. > [cid:1920d919-2658-4c49-9add-fa2148f9515e] > (4) Proof of Concept > You can test above cases by following codes. It needs 2 applications to trigger the bug. > (4.1) DRAM-overuse application > > #include <stdlib.h> > > int main(int argc, char* argv[]) > { > while(1) { > int *dummy = (int *) malloc (4096); > } > return 0; > } > > (4.2) Normal CUDA-using application > #include <cuda_runtime.h> > __global__ void cuda_function (float *input) > { > if (blockDim.x * blockIdx.x + threadIdx.x < 512) { > input[blockDim.x * blockIdx.x + threadIdx.x] += 1.0; > } > } > > int main(int argc, char* argv[]) > { > float *input; > float *comp = (float *) malloc(512 * sizeof(float)); > cudaMalloc(&input, 512*sizeof(float)); The strange thing here is that cudaMalloc doesn't use the kernel paths mentioned above. > cuda_function<<<16, 32>>>(input); > cudaMemcpy(&comp, input, 512 * sizeof(float), cudaMemcpyDeviceToHost); > return 0; > } > First, multiple DRAM-overuse applications should be executed background, so that they fill DRAM free area. > Second, While Swap in and out pages frequently occur in DRAM, execute Normal CUDA-using application multiple times. > Third, When CUDA-using application fails its execution due to the bug specified in (4) bug triggering flow, All following applications using CUDA driver cannot be executed normally. > 3. Keywords: device, driver, kernel, memory, allocation > 4. Kernel Version: From Old to Latest Kernel version, All versions are affected. > 5. Bug Fix. > Solution is simple. Clearing subsection_map’s mask in section_deactivate with correct nr_range counts, and deleting subsection_map unmasking role in memunmap_pages can be a solution > > References > [1] PixelVault: Using GPUs for Securing Cryptographic Operations, CCS, 2014, Giorgos Vasiliadis, et al. > [2] A framework for GPU-accelerated AES-XTS encryption in mobile devices, TENCON 2011, Mohammad Ahmed Alomari, et al. > [3] https://github.com/allenlee820202/Parallel-AES-Algorithm-using-CUDA > > Thanks, > Sihyun Roh. > ________________________________ > From: Greg KH <gregkh@xxxxxxxxxxxxxxxxxxx> > Sent: Tuesday, June 11, 2024 12:05 AM > To: �������� / �л� / ���⡤�������к� <sihyeonroh@xxxxxxxxx> > Cc: akpm@xxxxxxxxxxxxxxxxxxxx <akpm@xxxxxxxxxxxxxxxxxxxx>; security@xxxxxxxxxx <security@xxxxxxxxxx>; linux-mm@xxxxxxxxx <linux-mm@xxxxxxxxx> > Subject: Re: [Linux bug report] A bug breaking device drivers' fault isolation guarantees > > On Mon, Jun 10, 2024 at 02:58:16PM +0000, �������� / �л� / ���⡤�������к� wrote: >> Hi, >> I'm Sihyun Roh, a security researcher at Compsec Lab, Seoul National University. >> While testing NVIDIA GPU code, I discovered a minor mistake in the >> Linux kernel code. This issue can cause one process's fault to >> affect other processes, compromising the fault isolation >> guarantee. Given the potential security implications, I am >> forwarding this to the security team as well. >> If you have any questions, feel free to ask. >> Thank you for your efforts in maintaining the Linux kernel code. >> Sincerely, >> Sihyun > > > For obvious reasons we can't open unsolicited .pdf files. Can you send > this in text format? > > And if this is for an out-of-tree kernel driver, there's nothing we can > do about that :( > > thanks, > > greg k-h
