Hi XFS list,
I'm having some strange trouble with xfs_repair and xfs_metadump, which
I am hoping you can help with. I have an xfs filesystem which is backed
by an mdraid/LVM combination. Recently two drives failed in the RAID6,
then during the rebuild another disk failed. I was able to salvage the
array by using ddrescue to copy the failed drive to a new drive (only
8k were lost). Once I did that, I turned to xfs_repair to check that
the filesystem was okay.
So far, it has reported a large number of errors, but consistently gets
stuck during phase 3. I have used xfsprogs 3.1.7 as well as the latest
clone from git, and have also used -P and not used -P, with no luck. I
have saved stderr, but it is extremely large. Nothing obvious
distinguishes the last stderr messages from previous messages, where it
might indicate why xfs_repair has stalled. (I can post stderr or make
it available by HTTP if it helps.)
Next, I tried to take an xfs_metadump, as suggested by the man page.
2.9.4 reported a glibc error and got stuck, so I tried the latest
version, and got this (a few messages from xfs_metadump before the
error):
Copied 4653824 of 28351936 inodes (0 of 65 AGs)
xfs_metadump: badly aligned inode (start = 10366393)
Copied 4654656 of 28351936 inodes (0 of 65 AGs)
xfs_metadump: bad number of extents 189 in inode 10383406
xfs_metadump: bad number of extents 2003136628 in inode 10383425
Copied 4654912 of 28351936 inodes (0 of 65 AGs)
xfs_metadump: invalid size (139315) in symlink inode 10362975
Copied 4654976 of 28351936 inodes (0 of 65 AGs) *** glibc detected *** /root/xfsprogs-dev/db/xfs_db: free(): invalid next size (normal): 0x0000000000a1c000 ***
======= Backtrace: =========
/lib64/libc.so.6[0x7f36324d245f]
/lib64/libc.so.6(cfree+0x4b)[0x7f36324d28bb]
/root/xfsprogs-dev/db/xfs_db[0x419a93]
/root/xfsprogs-dev/db/xfs_db[0x41d6a3]
/root/xfsprogs-dev/db/xfs_db[0x41b182]
/root/xfsprogs-dev/db/xfs_db[0x41d772]
/root/xfsprogs-dev/db/xfs_db[0x41b182]
/root/xfsprogs-dev/db/xfs_db[0x41d466]
/root/xfsprogs-dev/db/xfs_db[0x417ee8]
/lib64/libc.so.6(__libc_start_main+0xf4)[0x7f363247d994]
/root/xfsprogs-dev/db/xfs_db[0x402519]
======= Memory map: ========
00400000-00473000 r-xp 00000000 08:02 97846 /root/xfsprogs-dev/db/xfs_db
00673000-00674000 rw-p 00073000 08:02 97846 /root/xfsprogs-dev/db/xfs_db
00674000-00687000 rw-p 00000000 00:00 0
009e8000-01120000 rw-p 00000000 00:00 0 [heap]
3a91a00000-3a91a04000 r-xp 00000000 08:02 384575 /lib64/libuuid.so.1.2
3a91a04000-3a91c03000 ---p 00004000 08:02 384575 /lib64/libuuid.so.1.2
3a91c03000-3a91c04000 rw-p 00003000 08:02 384575 /lib64/libuuid.so.1.2
3a91e00000-3a91e0d000 r-xp 00000000 08:02 384716 /lib64/libgcc_s-4.1.2-20080825.so.1
3a91e0d000-3a9200d000 ---p 0000d000 08:02 384716 /lib64/libgcc_s-4.1.2-20080825.so.1
3a9200d000-3a9200e000 rw-p 0000d000 08:02 384716 /lib64/libgcc_s-4.1.2-20080825.so.1
7f362ee86000-7f3632460000 r--p 00000000 fd:01 1351115 /usr/lib/locale/locale-archive
7f3632460000-7f36325ae000 r-xp 00000000 08:02 384393 /lib64/libc-2.5.so
7f36325ae000-7f36327ae000 ---p 0014e000 08:02 384393 /lib64/libc-2.5.so
7f36327ae000-7f36327b2000 r--p 0014e000 08:02 384393 /lib64/libc-2.5.so
7f36327b2000-7f36327b3000 rw-p 00152000 08:02 384393 /lib64/libc-2.5.so
7f36327b3000-7f36327b8000 rw-p 00000000 00:00 0
7f36327b8000-7f36327ce000 r-xp 00000000 08:02 384418 /lib64/libpthread-2.5.so
7f36327ce000-7f36329cd000 ---p 00016000 08:02 384418 /lib64/libpthread-2.5.so
7f36329cd000-7f36329ce000 r--p 00015000 08:02 384418 /lib64/libpthread-2.5.so
7f36329ce000-7f36329cf000 rw-p 00016000 08:02 384418 /lib64/libpthread-2.5.so
7f36329cf000-7f36329d3000 rw-p 00000000 00:00 0
7f36329d3000-7f36329da000 r-xp 00000000 08:02 384422 /lib64/librt-2.5.so
7f36329da000-7f3632bda000 ---p 00007000 08:02 384422 /lib64/librt-2.5.so
7f3632bda000-7f3632bdb000 r--p 00007000 08:02 384422 /lib64/librt-2.5.so
7f3632bdb000-7f3632bdc000 rw-p 00008000 08:02 384422 /lib64/librt-2.5.so
7f3632bdc000-7f3632bf8000 r-xp 00000000 08:02 448663 /lib64/ld-2.5.so
7f3632da0000-7f3632de4000 rw-p 00000000 00:00 0
7f3632df4000-7f3632df8000 rw-p 00000000 00:00 0
7f3632df8000-7f3632df9000 r--p 0001c000 08:02 448663 /lib64/ld-2.5.so
7f3632df9000-7f3632dfa000 rw-p 0001d000 08:02 448663 /lib64/ld-2.5.so
7fffbe85e000-7fffbe87f000 rw-p 00000000 00:00 0 [stack]
7fffbe899000-7fffbe89a000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
./xfsprogs-dev/db/xfs_metadump.sh: line 31: 20422 Aborted
/root/xfsprogs-dev/db/xfs_db$DBOPTS -F -i -p xfs_metadump -c "metadump$OPTS $2" $1
Here's xfs_info on the filesystem as mounted:
# xfs_info /mnt/sonoran/
meta-data=/dev/sonoranVG/sonoranLV isize=256 agcount=65,
agsize=61034784 blks
= sectsz=512 attr=0
data = bsize=4096 blocks=3906227200,
imaxpct=25
= sunit=0 swidth=0 blks, unwritten=1
naming =version 2 bsize=4096
log =internal bsize=4096 blocks=32768, version=1
= sectsz=512 sunit=0 blks, lazy-count=0
realtime =none extsz=4096 blocks=0, rtextents=0
Currently the filesystem is mountable, but I am fairly sure that there
are some errors. This is a snapshot backup server, so I could simply
start over without too much pain, but it'd be nice to be able to recover
the work I've done if possible. Alternatively, if I can at least have
xfs_repair finish, it might be possible to recreate the snapshot in less
time using the data that did survive.
If you need any more information please let me know. Thanks!
--keith
--
kkeller@xxxxxxxxxxxxxxxxxxxxxxxxxx
_______________________________________________
xfs mailing list
xfs@xxxxxxxxxxx
http://oss.sgi.com/mailman/listinfo/xfs
