On 05/27/2011 07:12 AM, Jon Tegner wrote: > A general question, suppose I have a parallel application, using mpi, > where really fast access to the file system is critical. > > Would it be stupid to consider a ram disk based setup? Say a 36 port QDR Ram disks won't work directly, due to lack of locking in tmpfs. You could create a tmpfs, then create a file that fills this up, then a loopback device pointing to that file, then build a file system atop that, and mount it. And then mount gluster atop that. Needless to say, all these layers significantly decrease performance and introduce inefficiencies. > infiniband with half of the ports connected to computational nodes and > the other half to gluster nodes? There may be other options, but the options are not going to be cheap/inexpensive. How fast, and by fast do you mean bandwidth and/or latency (e.g. streaming bandwidth or random IOPs)? What does your IO profile look like? You can get nodes that stream 4.6+ GB/s read, and 3.6+ GB/s writes for single readers/writers to single files. For MPI jobs with single readers/writers, this is good. For very large IO jobs where you need 10's of GB/s, you probably need a more specific design to your problem. Regards, Joe -- Joseph Landman, Ph.D Founder and CEO Scalable Informatics, Inc. email: landman at scalableinformatics.com web : http://scalableinformatics.com http://scalableinformatics.com/sicluster phone: +1 734 786 8423 x121 fax : +1 866 888 3112 cell : +1 734 612 4615
