Hi, Here is my dd command: dd if=/dev/zero of=/opt/laptop.bin bs=1G count=5 oflag=direct The file system of /opt is XFS: [root@localhost opt]# df -T | awk '{print $1,$2,$NF}' | grep "^/dev" /dev/mapper/cl-root xfs / /dev/sda1 xfs /boot And this is virtual machine on my dektop so I just use it to test fio tool only and there is no read/write running process. And I don’t think there is no problem between virtual machine and physical machine. I have also tested on 1 VM where I hosted using Virtuozzo. The page cache still increase. Thank you very much. Regards, Son -----Original Message----- From: Sitsofe Wheeler [mailto:sitsofe@xxxxxxxxx] Sent: Friday, April 21, 2017 1:33 PM To: Son Chu <son.ct@xxxxxxxxxxxxx> Cc: Srinivasa Chamarthy <chamarthy.raju@xxxxxxxxx>; fio@xxxxxxxxxxxxxxx Subject: Re: Does fio really bypass page cache with option direct=1? Hi, Can you show the exact dd command (options, "of" and all) you used? Also which filesystem is /opt/ running on and are you sure you're looking at the page cache just for the file you're writing (e.g. is something writing/reading elsewhere on the system at the same time if you're looking at the whole system page cache stats)? On 21 April 2017 at 06:48, Son Chu <son.ct@xxxxxxxxxxxxx> wrote: > > I still got page cache increasing even when I used those option. > Here is output without those option: > fio --filename=/opt/test --size=1G --direct=1 --sync=1 --rw=randrw > --bs=4k --numjobs=1 --iodepth=8 --runtime=60 --time_based --group_reporting --name=journal-test --invalidate=1 --gtod_reduce=1 --ioengine=libaio HITS MISSES DIRTIES RATIO BUFFERS_MB CACHE_MB > 337 0 0 100.0% 0 95 > 411 0 89005 100.0% 0 664 > 337 0 38526 100.0% 0 816 > 337 0 26605 100.0% 0 920 > 591 0 19906 100.0% 0 998 > 337 0 12283 100.0% 0 1046 > 457 0 11351 100.0% 0 1090 > 337 0 12398 100.0% 0 1138 > 337 0 13252 100.0% 0 1190 > 337 0 11876 100.0% 0 1237 > 337 0 8723 100.0% 0 1271 > 337 0 12844 100.0% 0 1321 > 337 0 4612 100.0% 0 1340 > 457 0 0 100.0% 0 1340 > 337 0 3 100.0% 0 1340 > 457 0 0 100.0% 0 1340 > 457 0 0 100.0% 0 1340 > 337 0 0 100.0% 0 1340 > 336 1 0 99.7% 0 316 > > And here is output with those option: > fio --filename=/opt/test --size=1G --direct=1 --sync=1 --rw=randrw > --bs=4k --numjobs=1 --iodepth=8 --runtime=60 --time_based --group_reporting --name=journal-test --invalidate=1 --gtod_reduce=1 --ioengine=libaio --create_on_open=1 --fallocate=none HITS MISSES DIRTIES RATIO BUFFERS_MB CACHE_MB > 337 0 0 100.0% 0 95 > 337 0 0 100.0% 0 95 > 337 0 0 100.0% 0 95 > 410 1 0 99.8% 0 316 > 337 0 9 100.0% 0 316 > 337 0 0 100.0% 0 316 > 337 0 3 100.0% 0 316 > 337 0 0 100.0% 0 316 > 337 0 0 100.0% 0 316 > 1828 0 10 100.0% 0 316 > 337 0 0 100.0% 0 316 > 337 0 0 100.0% 0 316 > 337 0 0 100.0% 0 316 > > As you can see the page cache still increase from 95 to 316 when using those option. > I don't understand why the dd command with option oflag=direct gave no page cache increasing while fio increased page cache. -- Sitsofe | http://sucs.org/~sits/ -- To unsubscribe from this list: send the line "unsubscribe fio" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html