On 2013/6/9 11:29, Zhao Hongjiang wrote: > On 2013/6/9 10:37, Zhao Hongjiang wrote: >> On 2013/6/9 6:30, Theodore Ts'o wrote: >>> On Sat, Jun 08, 2013 at 11:13:35AM +0800, Zhao Hongjiang wrote: >>>> >>>> I run xfstests #239 against mainline 3.10.0-rc3, unfortunately it failure in my QEMU. I run the >>>> case a hundred times, it certainly hit the failure several times. The failure msg is as follow: >>>> >>>> FSTYP -- ext4 >>>> PLATFORM -- Linux/x86_64 3.10.0-rc3-mainline >>>> >>>> generic/239 1s ... - output mismatch (see /home/zhj/xfstests/results/generic/239.out.bad) >>>> --- tests/generic/239.out 2013-06-07 22:04:09.000000000 -0400 >>>> +++ /home/zff/xfstests/results/generic/239.out.bad 2013-06-07 22:04:09.000000000 -0400 >>>> @@ -1,2 +1,515 @@ >>>> QA output created by 239 >>>> +hostname: Host name lookup failure >>> >>> OK, so this hostname failure is weird; I'm not sure what's causing >>> this, but this I presume unrelated to the failure at hand. >>> >>>> Silence is golden >>>> +0: 0x0 >>>> +1: 0x0 >>>> +2: 0x0 >>>> +3: 0x0 >>> >>> This indicates a problem. Test generic/239 is running >>> aio-dio-hole-filling-race.c, which submits an asynchronous, direct I/O >>> 4k write with a buffer containing non-zero contents to a sparse file, >>> and once the I/O has completed, it uses pread to read it back, using >>> the same descriptor, so it is doing the read using direct I/O. It >>> then checks to see if the read returns zero or not. >>> >>> The "XX: 0x0" lines indicates that buffer is zero, which implies that >>> somehow aio_complete() is getting called before the uninitialized to >>> initialized conversion is taking place. I'm not seeing how this is >>> happening, though, so I'm a bit puzzled. If there are any unwritten >>> extents, we don't call aio_complete() in ext4_end_io_dio(), but >>> instead the conversion is queued via a call to ext4_add_compete_io(), >>> and and aio_done() is only called on the iocb after the conversion is >>> complete. >>> >>> Can anyone see something that I might be missing? >>> >>> - Ted >>> >>> P.S. Zhao, what was the hardware that you using to find this failure? >> >> I'am use x86 and start a qumu-kvm to run the test. more informations: 8 Intel(R) Xeon(R) CPU E5620 @ 2.40GHz Host: scsi0 Channel: 02 Id: 01 Lun: 00 Vendor: LSI Model: MegaRAID SAS RMB Rev: 1.40 Type: Direct-Access ANSI SCSI revision: 05 >> >>> I'm not seeing it, but then again if the failure is only happening >>> once every few hundred runs that might explain it. I'm perhaps >> >> And as Christoph Hellwig said "the race is very easy to hit by using QEMU with >> native AIO support on a sparse image, and the result is filesystem corruption >> in the guest", i also run the test on the host, but nerver see the failure. > > Sorry, i run the test on host a hundred times again, there are six failures. > >> >>> wondering if we should add a mode to aio-dio-hole-filling-race.c which >>> allows it to try the race a large number of times, instead of just >>> once. >> >> This seems necessary, i'll give a patch for this. >> - Zhao >>> >>> P.P.S. One thought.... perhaps it might be useful to have a debug >>> mode where we use queue_delayed_work() to submit the conversion >>> request to the workqueue. It will of course make certain workloads >>> run slow as molasses, but it might expose some races so we can see >>> them more easily. >>> >>> . >>> >> > > > -- > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html > > . > -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
