Andres Freund wrote:
I think thats FUD. Sorry.
Yes, there's plenty of uncertainty and doubt here, but not from me. The test reports given so far have been so riddled with errors I don't trust any of them. As a counter example showing my expectations here, the "Testing Sandforce SSD" tests done by Yeb Havinga: http://archives.postgresql.org/message-id/4C4A9452.9070100@xxxxxxxxx followed the right method for confirming both write integrity and performance including pull the plug situations. Those I trusted. What Marti had posted, and what Phoronix investigated, just aren't that thorough.
Can you explain to me why fsync() may/should/could be *any* less reliable than O_DSYNC? On *any* platform. Or fdatasync() in the special way its used with pg, namely completely preallocated files.
If the Linux kernel has done extra work so that O_DSYNC writes are forced to disk including a cache flush, but that isn't done for just fdatasync() calls, there could be difference here. The database still wouldn't work right in that case, because checkpoint writes are still going to be using fdatasync.
I'm not sure what the actual behavior is supposed to be, but ultimately it doesn't matter. The history of the Linux kernel developers in this area has been so completely full of bugs and incomplete implementations that I am working from the assumption that we know nothing about what actually works and what doesn't without doing careful real-world testing.
I think the reasons why O_DSYNC is, especially, but not only, in combination with a small wal_buffers setting, slow in most circumstances are pretty clear.
Where's your benchmarks proving it then? If you're right about this, and I'm not saying you aren't, it should be obvious in simple bechmarks by stepping through various sizes for wal_buffers and seeing the throughput/latency situation improve. But since I haven't seen that done, this one is still in the uncertainty & doubt bucket too. You're assuming one of the observed problems corresponds to this theorized cause. But you can't prove a performance change on theory. You have to isolate it and then you'll know. So long as there are multiple uncertainties going on here, I don't have any conclusion yet, just a list of things to investigate that's far longer than the list of what's been looked at so far.
-- Greg Smith 2ndQuadrant US greg@xxxxxxxxxxxxxxx Baltimore, MD PostgreSQL Training, Services and Support www.2ndQuadrant.us "PostgreSQL 9.0 High Performance": http://www.2ndQuadrant.com/books -- Sent via pgsql-performance mailing list (pgsql-performance@xxxxxxxxxxxxxx) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
