On Tue, 6 Dec 2011 15:13:34 -0800 Yucong Sun (叶雨飞) <sunyucong@xxxxxxxxx> wrote: > On Tue, Dec 6, 2011 at 2:26 PM, NeilBrown <neilb@xxxxxxx> wrote: > > On Tue, 6 Dec 2011 14:01:14 -0800 Yucong Sun (叶雨飞) <sunyucong@xxxxxxxxx> > > wrote: > > > >> Hi, > >> > >> I recently setup raid10 on 4 physical disk and have a iscsi serve it > >> as a block device, and have been trying to tweak for performance. > >> > >> First thing I notice that MD seems to rely on page cache to flush > >> changes to disk, is there any way to turn that off so changes are > >> flushed to the disk? like O_FSYNC|O_DIRECT does? The reason I want to > >> turn it off is to understand the performance difference, I want to be > >> sure that page cache is truly acting as a write-back cache, I know one > >> can tune the dirty_* to control the cache flush, but I want to make > >> sure that it is actually doing what I think it does. > > > > Why do you think this? > > > > md/raid10 sends all request straight through to the relevant underlying > > device(s). > > reads are just passed straight down. > > Writes are duplicated (the request structure, not the data) and queued to a > > separate thread which does the actual write, but it is fairly direct. > > So I know there's page caching /flush involved because I watch > /proc/meminfo and see Dirty value growing up and After reach the > threshold, Write-back kicks in and wrote data. > So if as you said md does no page flushing, then it must because of > the iscsi software opens the device without O_DIRECT, so it uses page > cache which in turn flush data to MD, now it makes more sense. > > But for the md write, it's not SYNC write? meaning that after write > call with O_DIRECT to the md device returns, the data is still > possibility on the fly to the disk? how does having a bitmap plays in > between? does it work like ext3 jounal? after a power-loss, can we > expect a crash consistent data on the disk? When you want sync writes, you need to use fsync. When md writes the superblock or a bitmap page it uses SYNC and FLUSH writes to ensure they get to the media before the subsequent data write. > > Another thing to note is I found IO size on MD device is always 4K, > which is the page size, is that normal? just want to making sure this > isn't a bad behavior result from the iscsi software. It is normal in some cases. It depends a bit on the details of the underlying device. NeilBrown
Attachment:
signature.asc
Description: PGP signature
