On Thu, Sep 23, 2010 at 09:22:29AM +0200, Christian Herzog wrote: > > Dear all, > > we (Physics Dept. at ETH Zurich) are trying to set up a large file > server combo (two disk backends connected to a frontend by > Infiniband, all running Ubuntu 10.04) and keep getting XFS internal > error xfs_da_do_buf(2) messages when copying large amounts of data, > resulting in 'structure needs cleaning' warnings. We have tried a > lot of different kernels, iSCSI implementations, LVM configurations, > whatnot, but these errors persist. The setup right now looks like > this: > > 2 disk backends, each: Quad-Xeon X5550, 12G of RAM, 28T HW > SATA-RAID6 sliced into 2T chunks by LVM2 and exported via tgt > 1.0.0-2, Ubuntu 10.04 LTS, connected via Mellanox MHRH19B-XTR > Infiniband + ISER to > > 1 frontend Octo-Xeon E5520, 12G of RAM, open-iscsi 2.0.871 > initiator, Ubuntu 10.04 LTS. LMV2 stitches together the > 2T-iSCSI-LUNs and provides a 10T test XFS filesystem Out of curiousity, why are you using such a complex storage configuration? IMO, it is unneccessarily complex - you could easily do this (~30 drives) with a single server with a couple of external SAS JBOD arrays and SAS RAID controllers. That would give you the same performance (or better), with many fewer points of failure (both hardware and software), use less rack space, and probably be significantly cheaper.... Cheers, Dave. -- Dave Chinner david@xxxxxxxxxxxxx _______________________________________________ xfs mailing list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
