"rae l" <crquan@xxxxxxxxx> writes: > I found a problem with the vanilla kernel from 2.6.22 (include 23, 24): > > the situation is to test a customized linux distribution under heavy IO load: > 1. the client process initiates tens of POSIX threads (using > libpthread), each thread uses aio_write or aio_read(using librt) > operating > on one small file, then close it and write another small file; > the whole objective is to get a maxium throughput of small files, > by generating heavy aio stress on the system; > > I have tested the vanilla kernel 2.6.22/23/24.y, all these kernels > have the common problem: > 1. I use top, vmstat, and iostat to monitor the system performance, I > found that the iowait time of CPU is high at most time, > above 60%, and not stable while the client process running, > although the throughput is stable, the CPU iowait time not stable; > As a result of instability processes felt too long and unacceptable delay. > 2. in seldom testing cases, the system even stopped: with bi/bo are 0 > and iowait 100%, all writing processes are blocking with > uninterruptible state (D state in ps output), these cases are all > system running after several days writing, but cannot guarantee > to reproduce; > > First I suspect the filesystem or the storage medium have problems, I > tried ext2/ext3/reiserfs/xfs, for different filesytems and > SATA/SAS, IDE disks, and several commercial hard RAID card for > different storage medium, but the bad result remained; > > Then I tested 2.6.21.7 kernel, this results a stable iowait CPU time > (below 10%) but not so good throughput of small files; > > Now I think the improvements of IO effciency in the development of > 2.6.22 also caused the instability of iowait time, right? but how > to find out? Simple bisecting seems not work. Why doesn't bisecting work? Can you provide your test code so others can verify your findings? > by the way, a question is how to guarantee a kernel not regress under > heavy multi-thread aio writing load? ltp project seems not give the > answer: Well, providing your test code would be a step closer to achieving this guarantee. If it is scriptable, then there is a chance we could integrate it into the aio-dio regression test suite: http://git.kernel.org/?p=linux/kernel/git/zab/aio-dio-regress.git;a=summary Cheers, Jeff -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
