On 11/19/2013 05:26 PM, K.R Kishore wrote:
Hi I am trying to compare the performance of locally attached block device (SSD) with a network attached block device SSD)and I am seeing results for sequential reads and writes using that I cannot explain. The other results (random reads, writes etc) are as expected, i.e. local is better than remote. Here is my setup - Two machines connected back-to-back by a 10G link - Running RHEL 6.4 (Santiago), 2.6.32-358.6.1.el6.x86_64 - Running nbd v2.9.20 (http://nbd.sourceforge.net/) - Running fio v2.1.2 - Using identical SSD on both machines - Samsung 840 PRO, 128G - all 128G exported as rw volume I have my fio commands and output (only relevant portions) below. I cannot understand how the network device can have high throughput than local device. I see that when I use smaller block sizes to measure iops, the numbers are as expected (local > remote). Has anyone tried fio on nbd? does fio measure a transaction done when it sees the block-io request handed off to the virtual device and assume TCP will take care of completing the transaction? I can see that it might do so for posted operations such as writes, but reads? Any clues? thx, Kishore
278MB/s read bandwidth to a locally attached samsung 840 pro on 1M sequential reads is very low unless you have it accidentally plugged into a SATA 3Gb/s port instead of a 6Gb/s. I'd sort out why you're not seeing 500 MB+ on this as starting point for your investigation.
Also, Sequential performance probably isn't what you want to look at for a long latency block device (as opposed to without the network in the way) as io merging could become the dominant factor for performance even when using large block sizes to start.
your latency data from the runs looks funny too - with the NBD latency being lower than the locally attached on writes, but not for reads. that would seem to indicate there is some buffering going on in the system that you're not aware of that is making your results noisy (and confusing)
-- To unsubscribe from this list: send the line "unsubscribe fio" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
