Re: engine rbd broken after commit 2466096336bd0fbc1a94811aa338926af6baf42f

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 01/11/2016 07:03 AM, Jens Rosenboom wrote:
2016-01-11 12:09 GMT+01:00 Jens Rosenboom <j.rosenboom@xxxxxxxx>:
2016-01-08 12:15 GMT+01:00 Jens Rosenboom <j.rosenboom@xxxxxxxx>:
When running fio-2.3 with the rbd engine, the tests themselves run
fine, but instead of the results being printed, I get:

fio: mutex.c:163: fio_mutex_down: Assertion `mutex->magic ==
0x4d555445U' failed.

The command I am using is

fio --ioengine=rbd --clientname=admin --pool=test --rbdname=test1
--rw=randwrite --runtime=10 --ramp_time=None --numjobs=1 --direct=1
--bs=1048576B --iodepth=64 --name=test1

If I instead use another engine, everything is fine, e.g.:

fio --ioengine=libaio --filename=/dev/sdb1 --rw=randwrite --runtime=10
--ramp_time=None --numjobs=1 --direct=1 --bs=1048576B --iodepth=64
--name=test1

Going back to fio-2.2.13 makes the error go away and running a git
bisect leads to the commit mentioned in the subject as the culprit,
the error turns into a Segmentation Fault at that point, though.

Accidentally I found out that I can also reproduce the bug with libaio if I set

LD_PRELOAD=../jemalloc/lib/libjemalloc.so

compiled from https://github.com/jemalloc/jemalloc.git (tested with
3.6.0 and 4.0.4)

So it seems there is some use-after-free issue in this commit. I'll
try to dig into this a bit more when I get the time, but I won't be
sad if someone else is faster. ;)

In the standalone case, sk_out_key is used uninitialized, which may
happen to work fine until some other library also uses a pthread_key,
like it seems both librbd and libjemalloc do.

Attached is a patch that fixes this.

Thanks for reporting this! I merged a cleaner fix, we can just set up the key earlier and not have to worry about server vs non-server. Can you check if current -git works for you?


--
Jens Axboe

--
To unsubscribe from this list: send the line "unsubscribe fio" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html



[Index of Archives]     [Linux Kernel]     [Linux SCSI]     [Linux IDE]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux SCSI]

  Powered by Linux