I went thru the same hardware config gymnastics (confounded by bad disks, a bad controller, bad hotswap cages, and some SW config issues - see: http://maillists.uci.edu/mailman/public/uci-linux/2005-June/001067.html). The 3ware info is partially reported here: http://maillists.uci.edu/mailman/public/uci-linux/2005-June/001134.html and I believe I also sent a similar report to this list, but google doesn't show it. Here's the results of my bonnie tests on a similar system: 8x250GB WD SD series disks on a 3ware 9500S-8, on an IWILL 2x Opteron mobo, 4GB RAM, with XFS. The XFS parameters are with a 64K stripe to match the RAID card; other params more or less vanilla. Below are some bonnie timing results with differing filesystems (1 run with ext3, 3 with XFS with differnt file sizes - unwrap in an editor to compare in columns: XFS sand, 7000M,52315,99,104402,22,32242,8,32018,60,127998,20,435.5,0,80,1724,15,+++++, +++, 3248, 16,1718,15,+++++,+++,750, 4 ext3 sand, 7000M,40682,91, 47732,23,25432,9,38027,72,179352,27,311.1,0,80, 416,99,+++++,+++,53250,100, 423,99,+++++,+++,560,58 XFS sand, 7000M,50040,98,106324,24,33046,8,31269,59,112240,17,416.0,0,80,1930,17,+++++, +++, 3753, 17,1913,17,+++++,+++,449, 2 XFS sand, 15000M,51065,99,101659,24,26884,7,35344,69,141223,23,263.0,0,80,1666,14, +++++,+++, 4565, 22,1700,16,+++++,+++,793, 4 >From my reading (see URL above for resourcelist), XFS is quite bad for tiny files - we use it for very large files (>GB size); using XFS for this would generally be a bad thing. We do NOT get tremendous performance out of it; but the performance is much better than with ext3 and the CPU usage is lower, sometimes dramtically so. Real life experience with some benchmarks confirms this - we get approximately ~ the same real life thruput as we do on a IBM SP2 8way module with a direct attach disk. We are now considering adding a local PVFS2 system to a small cluster for very fast IO under MPI On Monday 25 July 2005 18:11, Gaspar Bakos wrote: > Dear all, > > The purpose of this email is twofold: > - to share the results of the many tests I performed with > a 3ware RAID card + RAID-5 + XFS, pushing for better file I/O, > - and to initiate some brainstorming on what parameters can be tuned for > getting a good performance out of this hardware under 2.6.* kernels. > > I started all these tests because the performance was quite poor, meaning > that the write speed was slow, the read speed was barely acceptable, and > the system load went very high (10.0) during bonnie++ tests. > My questions are marked below with "Q". <clipped> - To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
