[BUG] Invalid return address of mmap() followed by mbind() in multithreaded context

[Vasileios Karakasis <bkk@xxxxxxxxxxxxxxxxx>]

Hi,

I am implementing a multithreaded numa aware code where each thread
mmap()'s an anonymous private region and then mbind()'s it to its local
node. The threads are performing a series of such mmap() + mbind()
operations. My program crashed with SIGSEGV and I noticed that mmap()
returned an invalid address.

I am sending you a simple program that reproduces the error. The program
creates two threads and each thread starts allocating pages and then
binds them to the local node 0. After a number of iterations the program
crashes as it tries to dereference the address returned by mmap(). The
bug doesn't come up when using a single thread, neither when using only
mmap().

I am running a 2.6.39.1 kernel on a 64-bit dual-core machine, but I
tracked this bug back down to the 2.6.34.9 version.

This bug also affects libnuma.

Regards,
-- 
V.K.


#include <assert.h>
#include <sys/mman.h>
#include <pthread.h>
#include <numaif.h>

#define NR_ITER 10240
#define PAGE_SIZE 4096

void *thread_func(void *args)
{
    unsigned char *addr;
    int err, i;
    unsigned long node = 0x1;

    for (i = 0; i < NR_ITER; i++) {
        addr = mmap(0, PAGE_SIZE, PROT_READ | PROT_WRITE,
                    MAP_ANONYMOUS | MAP_PRIVATE, 0, 0);
        if (addr == (void *) -1) {
            assert(0 && "mmap failed");
        }
        *addr = 0;

        err = mbind(addr, PAGE_SIZE, MPOL_BIND, &node, sizeof(node), 0);
        if (err < 0) {
            assert(0 && "mbind failed");
        }
    }
    return (void *) 0;
}

int main(void)
{
    pthread_t thread;
    pthread_create(&thread, NULL, thread_func, NULL);
    thread_func(NULL);
    pthread_join(thread, NULL);
    return 0;
}

Attachment: 0x17A67A9C.asc
Description: application/pgp-keys

Attachment: signature.asc
Description: OpenPGP digital signature