Josh, Thanks for the feedback. Given the prevalence of SSDs/VMs, it would be useful to start collecting stats/tuning for different operating systems for things like fsync (and possibly bonnie++/dd). If the community is interested, I¹ve got a performance lab that I¹d be willing to help run tests on. Having this information would only improve our ability to support our customers. M. On 4/23/14, 12:04 PM, "Josh Berkus" <josh@xxxxxxxxxxxx> wrote: >Mel, > >> I was given anecdotal information regarding HFS+ performance under OSX >>as >> being unsuitable for production PG deployments and that pg_test_fsync >> could be used to measure the relative speed versus other operating >>systems > >You're welcome to identify your source of anecdotal evidence: me. And >it's based on experience and testing, although I'm not allowed to share >the raw results. Let's just say that I was part of two different >projects where we moved from using OSX on Apple hardware do using Linux >on the *same* hardware ... because of intolerable performance on OSX. >Switching to Linux more than doubled our real write throughput. > >> Compare file sync methods using one 8kB write: >> (in wal_sync_method preference order, except fdatasync >> is Linux's default) >> open_datasync 8752.240 ops/sec 114 >>usecs/op >> fdatasync 8556.469 ops/sec 117 >>usecs/op >> fsync 8831.080 ops/sec 113 >>usecs/op >========================================================================== >== >> fsync_writethrough 735.362 ops/sec 1360 >>usecs/op >========================================================================== >== >> open_sync 8967.000 ops/sec 112 >>usecs/op > >fsync_writethrough is the *only* relevant stat above. For all of the >other fsync operations, OSX is "faking it"; returning to the calling >code without ever flushing to disk. This would result in data >corruption in the event of an unexpected system shutdown. > >Both OSX and Windows do this, which is why we *have* fsync_writethrough. > Mind you, I'm a little shocked that performance is still so bad on >SSDs; I'd attributed HFS's slow fsync mostly to waiting for a full disk >rotation, but apparently the primary cause is something else. > >You'll notice that the speed of fsync_writethrough is 1/4 that of >comparable speed on Linux. You can get similar performance on Linux by >putting your WAL on a ramdisk, but I don't recommend that for production. > >But: things get worse. In addition to the very slow speed on real >fsyncs, HFS+ has very primitive IO scheduling for multiprocessor >workloads; the filesystem was designed for single-core machines (in >1995!) and has no ability to interleave writes from multiple concurrent >processes. This results in "stuttering" as the IO system tries to >service first one write request, then another, and ends up stalling >both. If you do, for example, a serious ETL workload with parallelism >on OSX, you'll see that IO throughput describes a sawtooth from full >speed to zero, being near-zero about half the time. > >So not only are fsyncs slower, real throughput for sustained writes on >HFS+ are 50% or less of the hardware maximum in any real multi-user >workload. > >In order to test this, you'd need a workload which required loading and >sorting several tables larger than RAM, at least two in parallel. > >In the words of the lead HFS+ developer, Don Brady: "Since we believed >it was only a stop gap solution, we just went from 16 to 32 bits. Had we >known that it would still be in use 15 years later with multi-terabyte >drives, we probably would have done more design changes!" > >HFS+ was written in about 6 months, and is largely unimproved since its >release in 1995. Ext2 doesn't perform too well, either; the difference >is that Linux users have alternative filesystems available. > >-- >Josh Berkus >PostgreSQL Experts Inc. >http://pgexperts.com > > >-- >Sent via pgsql-performance mailing list (pgsql-performance@xxxxxxxxxxxxxx) >To make changes to your subscription: >http://www.postgresql.org/mailpref/pgsql-performance -- Sent via pgsql-performance mailing list (pgsql-performance@xxxxxxxxxxxxxx) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
