On 3/28/2013 4:45 PM, Ralf Gross wrote: > Stan Hoeppner schrieb: > Snapshots are possible with RDM in virtual compatibily mode, not > physical mode (> 2 TB). So 2TB is the kicker here. I haven't used ESX since 3.x, and none of our RDMs back then were close to 2TB. IIRC our largest was 500GB. >> VMFS volumes are not intended for high performance IO. Unless things >> have changed recently, VMware has always recommended housing only OS >> images and the like in VMDKs, not user data. They've always recommended >> using RDMs for everything else. IIRC VMDKs have a huge block (sector) >> size, something like 1MB. That's going to make XFS alignment difficult, >> if not impossible. > > I can't remember that I've every found this recommendation on a vmware > page. > > http://blogs.vmware.com/vsphere/2013/01/vsphere-5-1-vmdk-versus-rdm.html If you drill down through that you find this: http://www.vmware.com/files/pdf/performance_char_vmfs_rdm.pdf RDMs have better large sequential performance, and lower CPU burn than VMDKs. The OP mentioned "compute node" in his post, which suggests an HPC application workload, which suggests large sequential IO. Also note that VMware is Microsoft centric so they always run their tests using an MS Server guest. Also note they always test with tiny volumes, in this case 20GB. NTFS isn't going to have any trouble at this size, but at say 20TB it probably will and these published results would likely be quite different at that scale. XFS performance characteristics on a 2TB or 20TB or ?? TB volume will likely be substantially different than NTFS. Their tests show 5-8% lower CPU burn for RDM vs VMDK. Not a huge difference, but again they're testing only 20GB. >> I cannot stress emphatically enough that you should not stitch 2TB VMDKs >> together and use them in the manner you described. This is a recipe for >> disaster. Find another solution. > > I'm seeing more and more requests for VMs with large disks lately in my > env. Right now the max. is ~2 TB. I'm also thinking about where to go, > > 2 TB ist only possible with pRDMs which can't be snapshotted. You > have to use the snapshot features of your storage array. And more and more folks are using midrange FC/iSCSI arrays that don't have snapshot features, others are using DAS with RAID HBAs, in both cases forcing them to rely on ESX snapshots. Sounds like VMware needs to bump this artificial 2TB limit quite a bit higher. -- Stan _______________________________________________ xfs mailing list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
