On 21/12/11 14:26, Jeff White wrote: > I'm going to run into the 'lots of small files on replica sucks' problem. Good luck! I found it is not the performance so much that is the problem, but the consequential system load on the bricks with many small files. I found that copying 3 large IMAP accounts simultaneously to a new server having Maildirs on gfs (one brick hosted on a different server, 16-core 32GB machine with two 1.5TB SATA drives configured in RAID 10, gigabit ethernet) created such I/O load on the brick server as to bring it to it's knees (uptime load numbers around 80-90) despite CPU load closer to 5%. A couple of very small VMs running on the brick server were just completely choked. I tried a couple of gfs tuning options with no observable difference. In my case it was an quick conclusion to just use an IMAP proxy and stick to local storage, though I did wonder afterwards if perhaps the local I/O got choked because there were 16 cores to run gluster threads in parallel. If it is possible to restrict gluster's thread count, maybe it would have worked better? And I guess with greater distribution across bricks the load would be shared between the disk I/O potential of many more machines. I'd be interested to hear more experienced views on these things. -Will
