On 9/26/11, Robert Krig <robert at bitcaster.de> wrote: > I guess the question to ask here is, do you need a lot of read/write > performance for your application, or is redundancy and synchronisation > more important? All would be nice but of course I know in the real world, there has to be some compromise. For the client's setup, I don't think performance is the #1 factor but at the very least the system has to be able to sustain 8MB/s of transfers (going by their 10Mbps~20Mbps connection, and x2 due to replication required) on bonded 1G ethernet. Just as important is the latency, which was the key problem pointed out in the rackerhacker blog, 3~4 seconds latency is bad. I'd rather have 0.5 second latency with 5MB/s than 5 seconds lag with 50MB/s performance. More importantly is the data integrity and redundancy, the former being more important since redundant corrupted data is useless. Which is why the bug about the corruption of dynamically generated/edited files is a concern. > As you can guess, rsync is not so good with lots of small files, at > least not THAT many small files, so with a 10Gigabit ethernet > connection, on the small files we got about 10-30 megabytes per second. 10~30MB/s is more than OK for me. However, you're on 10G while my client has a budget I need to work within so bonded 1G with VLAN is probably the best I can do. Any idea/data on how much an impact that might make? > Of course, regardless of what other people might have experienced. Your > best bet ist to test it with your own equipment. There are so many > variables between differing distros, kernels, optimisations, and > hardware, it's hard to guarantee any kind of minimum performance. Unfortunately, I need to make an good estimate on the best file system to go with in order to plan and go to them with a budget for the hardware before any testing could be done. While I could try to put together a test network with VM on our spare hardware, there are just too many bottlenecks and variables introduced that make such tests useless except as proof of concept that the setup is sane and would work.
