On 01/10/2011 07:43 PM, Christopher Hawkins wrote: > I am testing Infiniband for the first time. It seems that I should be > able to get a lot more speed than I am with some pretty basic tests. > Maybe someone running Infiniband can confirm that what I am seeing is > way out of line, and/or help diagnose? What is the underlying disk speed? Try this: # (on bravo) dd if=/dev/zero of=/cluster/shadow/big.file bs=1M count=20k This will write a 20GB file to the same partition. We need to see how fast that write is (outside of cache) Do the same test on the other machine. Infiniband won't make slow disks fast. So unless you are already getting a gigabyte or two per second (we do on our units), adding Infiniband won't get you any significant additional performance. Put another way, you are bound by the slowest aspect of your storage system. What are your underlying disks? Are these individual drives? RAID? What sort of RAID if it is a RAID? How is it organized? We tune our systems pretty hard, so we start with 2+GB/s for TB sized files before we ever touch the next stack up. Each additional stack you traverse takes performance away (you lose it in stack inefficiency). So you should start out with as fast a system as you possibly can, so you don't lose too much performance. -- Joseph Landman, Ph.D Founder and CEO Scalable Informatics, Inc. email: landman at scalableinformatics.com web : http://scalableinformatics.com http://scalableinformatics.com/sicluster phone: +1 734 786 8423 x121 fax : +1 866 888 3112 cell : +1 734 612 4615
