Naoya Horiguchi <n-horiguchi@xxxxxxxxxxxxx> writes: > Hi, > > On Thu, Aug 12, 2010 at 09:42:21AM -0400, Jeff Moyer wrote: >> Naoya Horiguchi <n-horiguchi@xxxxxxxxxxxxx> writes: >> >> > Basically it is user's responsibility to take care of race condition >> > related to direct I/O, but some events which are out of user's control >> > (such as memory failure) can happen at any time. So we need to lock and >> > set/clear PG_writeback flags in dierct I/O code to protect from data loss. >> >> Did you do any performance testing of this? If not, please do and >> report back. I'm betting users won't be pleased with the results. > > Here is the result of my direct I/O benchmarck, which mesures the time > it takes to do direct I/O for 20000 pages on 2MB buffer for four types > of I/O. Each I/O is issued for one page unit and each number below is > the average of 25 runs. > > with patchset 2.6.35-rc3 > Buffer I/O type average(s) STD(s) average(s) STD(s) diff(s) > hugepage Sequential Read 3.87 0.16 3.88 0.20 -0.01 > Sequential Write 7.69 0.43 7.69 0.43 0.00 > Random Read 5.93 1.58 6.49 1.45 -0.55 > Random Write 13.50 0.28 13.41 0.30 0.09 > anonymous Sequential Read 3.88 0.21 3.89 0.23 -0.01 > Sequential Write 7.86 0.39 7.80 0.34 0.05 > Random Read 7.67 1.60 6.86 1.27 0.80 > Random Write 13.50 0.25 13.52 0.31 -0.01 > > From this result, although fluctuation is relatively large for random read, > differences between vanilla kernel and patched one are within the deviations and > it seems that adding direct I/O lock makes little or no impact on performance. First, thanks for doing the testing! > And I know the workload of this benchmark can be too simple, so please > let me know if you think we have another workload to be looked into. Well, as distasteful as this sounds, I think a benchmark that does I/O to partial pages would show the problem best. And yes, this does happen in the real world. ;-) So, sequential 512 byte or 1k or 2k I/Os, or just misalign larger I/Os so that two sequential I/Os will hit the same page. I believe you can use fio to generate such a workload; see iomem_align in the man page. Something like the below *might* work. If not, then simply changing the bs=4k to bs=2k and getting rid of iomem_align should show the problem. Cheers, Jeff [global] ioengine=libaio iodepth=32 bs=4k direct=1 size=2g overwrite=1 [test1] rw=write iomem_align=2k -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxxx For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>