On 07/05/12 17:18, J. Bruce Fields wrote: > On Mon, May 07, 2012 at 01:59:42PM +0000, Daniel Pocock wrote: >> >> >> On 07/05/12 09:19, Daniel Pocock wrote: >>> >>>>> Ok, so the combination of: >>>>> >>>>> - enable writeback with hdparm >>>>> - use ext4 (and not ext3) >>>>> - barrier=1 and data=writeback? or data=? >>>>> >>>>> - is there a particular kernel version (on either client or server side) >>>>> that will offer more stability using this combination of features? >>>> >>>> Not that I'm aware of. As long as you have a kernel > 2.6.29, then LVM >>>> should work correctly. The main problem is that some SATA hardware tends >>>> to be buggy, defeating the methods used by the barrier code to ensure >>>> data is truly on disk. I believe that XFS will therefore actually test >>>> the hardware when you mount with write caching and barriers, and should >>>> report if the test fails in the syslogs. >>>> See http://xfs.org/index.php/XFS_FAQ#Write_barrier_support. >>>> >>>>> I think there are some other variations of my workflow that I can >>>>> attempt too, e.g. I've contemplated compiling C++ code onto a RAM disk >>>>> because I don't need to keep the hundreds of object files. >>>> >>>> You might also consider using something like ccache and set the >>>> CCACHE_DIR to a local disk if you have one. >>>> >>> >>> >>> Thanks for the feedback about these options, I am going to look at these >>> strategies more closely >>> >> >> >> I decided to try and take md and LVM out of the picture, I tried two >> variations: >> >> a) the boot partitions are not mirrored, so I reformatted one of them as >> ext4, >> - enabled write-cache for the whole of sdb, >> - mounted ext4, barrier=1,data=ordered >> - and exported this volume over NFS >> >> unpacking a large source tarball on this volume, iostat reports write >> speeds that are even slower, barely 300kBytes/sec > > How many file creates per second? > I ran: nfsstat -s -o all -l -Z5 and during the test (unpacking the tarball), I see numbers like these every 5 seconds for about 2 minutes: nfs v3 server total: 319 ------------- ------------- -------- nfs v3 server getattr: 1 nfs v3 server setattr: 126 nfs v3 server access: 6 nfs v3 server write: 61 nfs v3 server create: 61 nfs v3 server mkdir: 3 nfs v3 server commit: 61 I decided to expand the scope of my testing too, I want to rule out the possibility that my HP Microserver with onboard SATA is the culprit. I set up two other NFS servers (all Debian 6, kernel 2.6.38): HP Z800 Xeon workstation Intel Corporation 82801 SATA RAID Controller (operating as AHCI) VB0250EAVER (250GB 7200rpm) Lenovo Thinkpad X220 Intel Corporation Cougar Point 6 port SATA AHCI Controller (rev 04) SSDSA2BW160G3L (160GB SSD) Both the Z800 and X220 run as NFSv3 servers Each one has a fresh 10GB logical volume formatted ext4, mount options: barrier=1,data=ordered write cache (hdparm -W 1): enabled Results: NFS client: X220 NFS server: Z800 (regular disk) iostat reports about 1,000kbytes/sec when unpacking the tarball This is just as slow as the original NFS server NFS client: Z800 NFS server: X220 (SSD disk) iostat reports about 22,000kbytes/sec when unpacking the tarball It seems that buying a pair of SSDs for my HP MicroServer will let me use NFS `sync' and enjoy healthy performance - 20x faster. However, is there really no other way to get more speed out of NFS when using the `sync' option? -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
