On 1/25/21 12:13 PM, Eric Sandeen wrote: > When running a job like this, with external engines enabled: > > [global] > ioengine=pmemblk > directory=/mnt/test > thread=1 > direct=1 > iodepth=1 > > [pmemblk-job2] > filename=pmemblk-smallrw,4096,17 > rw=readwrite > > [pmemblk-job2] > filename=pmemblk-randrw,4096,64 > rw=randrw > > it segfaults if the first thread to finish is the one that dlopened the engine, > and the 2nd thread has everything ripped out from under it: > > # fio --debug=io pmemblk.fio > ... > io 18361 close ioengine pmemblk > io 18362 prep: io_u 0x285e1c0: off=0x2108000,len=0x1000,ddir=1,file=/mnt/test/pmemblk-randrw,4096,64 > io 18362 queue: io_u 0x285e1c0: off=0x2108000,len=0x1000,ddir=1,file=/mnt/test/pmemblk-randrw,4096,64 > io 18361 free ioengine pmemblk > io 18362 complete: io_u 0x285e1c0: off=0x2108000,len=0x1000,ddir=1,file=/mnt/test/pmemblk-randrw,4096,64 > io 18362 fill: io_u 0x285e1c0: off=0x1bcc000,len=0x1000,ddir=0,file=/mnt/test/pmemblk-randrw,4096,64 > io 18361 ioengine pmemblk unregistered > io 18362 prep: io_u 0x285e1c0: off=0x1bcc000,len=0x1000,ddir=0,file=/mnt/test/pmemblk-randrw,4096,64 > io 18362 queue: io_u 0x285e1c0: off=0x1bcc000,len=0x1000,ddir=0,file=/mnt/test/pmemblk-randrw,4096,64 > io 18362 complete: io_u 0x285e1c0: off=0x1bcc000,len=0x1000,ddir=0,file=/mnt/test/pmemblk-randrw,4096,64 > io 18362 fill: io_u 0x285e1c0: off=0xe91000,len=0x1000,ddir=0,file=/mnt/test/pmemblk-randrw,4096,64 > io 18362 prep: io_u 0x285e1c0: off=0xe91000,len=0x1000,ddir=0,file=/mnt/test/pmemblk-randrw,4096,64 > io 18362 queue: io_u 0x285e1c0: off=0xe91000,len=0x1000,ddir=0,file=/mnt/test/pmemblk-randrw,4096,64 > io 18362 complete: io_u 0x285e1c0: off=0xe91000,len=0x1000,ddir=0,file=/mnt/test/pmemblk-randrw,4096,64 > io 18362 fill: io_u 0x285e1c0: off=0x3413000,len=0x1000,ddir=0,file=/mnt/test/pmemblk-randrw,4096,64 > io 18362 prep: io_u 0x285e1c0: off=0x3413000,len=0x1000,ddir=0,file=/mnt/test/pmemblk-randrw,4096,64 > io 18362 queue: io_u 0x285e1c0: off=0x3413000,len=0x1000,ddir=0,file=/mnt/test/pmemblk-randrw,4096,64 > Segmentation fault (core dumped) > > Fix this by keeping the dlhandle on the io engine itself, not the thread, > then explicitly dlopen for every thread that asks for the engine, and let > dlopen/dlclose do refcounting as designed. > > I think this is right, and it fixes it for me, but good review would be wise! > > Also: maybe the 2 patches could be collapsed, though I /think/ this makes review > a bit easier and doesn't add any regression at patch1 ... Applied them separately, thanks. -- Jens Axboe
