Hi. I really fancy GlusterFS since it truly feels like a system which can scale out nicely on the storage side which frankly most solutions don't. I've been working with big HP SAN's and NetApp. Both solutions was exported as NFS shares and both solutions had huge problems at peak traffic time even though it was high-end stuff. Anyway I'm not gonna walk down my memory lane of NFS Storage for two long but want to ask some storage experts a question. I'm quite good at scaling webapps but I'm not experienced enough to scale storage, that's why I'm turning here. I will use GlusterFS as a means for storing and accessing crawled and indexed data of let's say 1 billion entries. The data will consist of two types. 1. Big index files ~x Gig each 2. Many small files in a huge amount of directories. I will go for many cheap SATA disks since I think SAS just won't do storage wise. Does anyone have experience of good price worthy storage enclosures in cojunction with servers (Example Dell 1950 + MD1000 or HPDL160 + HP MSA60 ) or more complete solutions like Dell AFX150 or HP DL385? Does anyone as well know if it is better to go for separating enclosures from servers to get best-of-the-best of the two worlds ? I really would like a solution where I just can buy another 2 enclosures and 4 GlusterFS servers whenever performance or storage constraints kicks in. Scale out easily, cheaply, always increasing performance is my mantra :) Finally what tools would suite to test zillions of small files ? Bonnie++ ? Fewer big files ? Still Bonnie++ or perhaps IOZone ? Kindly /Marcus Herou -- Marcus Herou CTO and co-founder Tailsweep AB +46702561312 marcus.herou@xxxxxxxxxxxxx http://www.tailsweep.com/ http://blogg.tailsweep.com/
