On Thu, 25 Oct 2012, Theodore Ts'o wrote:
Or does rsyslog *really* need to issue an fsync after each log message? Or could it batch updates so that every N seconds, it flushes writes to the disk?
In part this depends on how paranoid the admin is. By default rsyslog doesn't do fsyncs, but admins can configure it to do so and can configure the batch size.
However, what I'm talking about here is not normal message traffic, it's the case where the admin has decided that they don't want to use the normal inmemory queues, they want to have the queues be on disk so that if the system crashes the queued data will still be there to be processed after the crash (In addition, this can get used to cover cases where you want queue sizes larger than your available RAM)
In this case, the extreme, and only at the explicit direction of the admin, is to fsync after every message.
The norm is that it's acceptable to loose the last few messages, but loosing a chunk out of the middle of the queue file can cause a whole lot more to be lost, passing the threshold of acceptable.
Sometimes, the answer is not to try to create exotic database like functionality in the file system --- the answer is to be more intelligent at the application leyer. Not only will the application be more portable, it will also in the end be more efficient, since even with the most exotic database technologies, the most efficient transactional commit is the unneeded commit that you optimize away at the application layer.
I agree, this is why I'm trying to figure out the recommended way to do this without needing to do full commits.
Since in most cases it's acceptable to loose the last few chunks written, if we had some way of specifying ordering, without having to specify "write this NOW", the solution would be pretty obvious.
David Lang -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
