On Wed, Oct 24, 2012 at 04:19:28PM -0500, Wayne Walker wrote: > On 10/12/2012 07:14 PM, Dave Chinner wrote: > <snip> > >And SB/AGF 3 and 4 are ok, too. So, the filesystem headers just > >beyond the 2TB offset are zero. That tends to point to a block > >device problem, as an offset of 2TB is where a 32 bit sector count > >will overflow (i.e. 2^32). Next step is to run blktrace/blkparse > >on the cp workload that generates the error to see if anything > >actually writes to the 2TB offset region, and if so, where it > >comes from. Probably best to compress the resultant blkparse > >output file - it might be quite large but the text will compress > >well. Cheers, Dave. > > Dave, > > Thank you for your help. > > 10 MB .gz file at http://rx-7.bybent.com/blktrace.sde1.out.gz > > What I can see seems to have most of the writes are around 2^31. Which is 2^31 sectors of just above 1TB. i.e. writing the data into AG #1. The writes stop a short way into #AG1. The filesystem does not issue any writes to the AG#2 headers, only a single read. IOWs, the filesystem is not overwriting it's own metadata during the workload, so that implies a problem at a lower storage layer.... IOWs, I can't see the filesystem doing anything wrong here. FWIW, can you pattern the block device around the 2TB offset and re-run the test, and see if the sb/agf 2 are zeroed via xfs_db after the failure occurs? i.e. do something like: # xfs_io -F -f -c "pwrite 2047g 2g" -c fsync /dev/sde1 the mkfs, run xfs_db to dump sb 2 and agf 2, then run the test and dump sb 2/agf 2 again after the test? use the same xfs-db scripts as the previous time (i.e. including the drop caches commands) if you can. Cheers, Dave. -- Dave Chinner david@xxxxxxxxxxxxx _______________________________________________ xfs mailing list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
