2011/10/25 Sage Weil <sage@xxxxxxxxxxxx>: > On Tue, 25 Oct 2011, Josef Bacik wrote: >> At this point it seems like the biggest problem with latency in ceph-osd >> is not related to btrfs, the latency seems to all be from the fact that >> ceph-osd is fsyncing a block dev for whatever reason. > > There is one place where we sync_file_range() on the journal block device, > but that should only happen if directio is disabled (it's on by default). > > Christian, have you tweaked those settings in your ceph.conf? It would be > something like 'journal dio = false'. If not, can you verify that > directio shows true when the journal is initialized from your osd log? > E.g., > > 2011-10-21 15:21:02.026789 7ff7e5c54720 journal _open dev/osd0.journal fd 14: 104857600 bytes, block size 4096 bytes, directio = 1 > > If directio = 1 for you, something else funky is causing those > blkdev_fsync's... I've looked it up in the logs - directio is 1: Oct 25 17:20:16 os00 osd.000[1696]: 7f0016841740 journal _open /dev/vg01/lv_osd_journal_0 fd 15: 17179869184 bytes, block size 4096 bytes, directio = 1 Regards, Christian -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
