Thanks John for the insight. If I understand you right, even if programs used fsync() if there is sufficient clients and/or workload then the journal logs will be flushed out to disks faster then the tick. This will be helpful to know in case some of the programs we use happen to use fsync() for ACD's sake. On Tue, May 26, 2015 at 10:40 AM, John Spray <john.spray@xxxxxxxxxx> wrote: > > > On 26/05/2015 15:50, Barclay Jameson wrote: >> >> Thank you for the great explanation Zheng! That definitely shows what >> I was seeing with the bonnie++ test. What bad things would happen if I >> modified the config option mds_tick_interval to flush the journal to a >> second or less? > > > The MDS does various pieces of housekeeping according to that interval, so > setting it extremely low will cause some CPU cycles to be wasted, and > flushing the log more often will cause a larger number of smaller IOs to get > generated. I would be very surprised if decreasing it to approx 1s was > harmful though. > > On a busy real world system, other metadata operations will often drive log > writes through faster than waiting for a tick. > >> Does this also mean any custom code written should >> avoid use of fsync() if writing a large number of files? > > > You should call it only when your application requires it for consistency, > and always expect it to be a high latency operation. Add up the latency from > your client to your server and from the server to the disk, and the length > of the IO queue on the disk, and then the return leg -- that is the > *minimum* time you should expect to wait for an fsync. > > For example, a real world workload creating N files in a directory would > hopefully call fsync on the directory once at the end, rather than in > between every file, unless you really do need to be sure that the dentry for > the preceding file will be persistent before you start writing the next > file. > > Sometimes it's easier to reason about it in terms of concurrency: if you > have a bunch of IOs that you could safely run in parallel in a thread each, > then you shouldn't be fsyncing between them, just at the point you join > them. > > John -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
