On 1/5/13 7:47 AM, 符永涛 wrote: > Dear xfs experts, > > We're running glusterfs over top of xfs and recently we have > encountered xfs filesystem crash two times on two different servers. > Can you kindly help to give me some insight of how to debug this kind > of failure? Please include more information about your system; at a minimum, what kernel are you running? See also: http://xfs.org/index.php/XFS_FAQ#Q:_What_information_should_I_include_when_reporting_a_problem.3F for a list of all relevant information to gather when reporting a problem. If this is a distribution kernel, in particular if you are talking about Red Hat Storage on Red Hat Enterprise Linux, you should probably contact your Red Hat support people first. > Bellow is xfs filesystem crash messages: > server1: > Dec 28 16:55:01 localhost kernel: XFS (dm-0): xfs_iunlink_remove: > xfs_inotobp() returned error 22. that is EINVAL, returned from xfs_inotbp, which tries to map an inode number to the buffer containing the on-disk version of the inode. It would take some debugging (and at least some more information about your system) to know where the EINVAL came from. > Dec 28 16:55:01 localhost kernel: XFS (dm-0): xfs_inactive: xfs_ifree > returned error 22 > Dec 28 16:55:01 localhost kernel: XFS (dm-0): So freeing the inode encountered a runtime error (EINVAL) and then: > xfs_do_force_shutdown(0x1) called from line 1184 of file > fs/xfs/xfs_vnodeops.c. Return address = 0xffffffffa029f20a > Dec 28 16:55:01 localhost kernel: XFS (dm-0): I/O Error Detected. > Shutting down filesystem the filesystem shut down as a result. -Eric > Dec 28 16:55:01 localhost kernel: XFS (dm-0): Please umount the > filesystem and rectify the problem(s) > Dec 28 16:55:20 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 16:55:50 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 16:56:20 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 16:56:50 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 16:57:20 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 16:57:50 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 16:58:20 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 16:58:50 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 16:59:20 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 16:59:50 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 17:00:20 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 17:00:50 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 17:01:20 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 17:01:50 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 17:02:20 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 17:02:50 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 17:03:20 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 17:03:50 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 17:04:20 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 17:04:20 localhost sshd[20609]: Accepted publickey for > fuyongtao from 10.1.12.99 port 56514 ssh2 > Dec 28 17:04:50 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 17:05:20 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 17:05:50 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 17:06:20 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 17:06:50 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 17:07:20 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 17:07:50 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 17:08:20 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 17:08:50 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 17:08:55 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 17:08:55 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned. > Dec 28 17:08:55 localhost kernel: XFS (dm-0): > xfs_do_force_shutdown(0x1) called from line 1056 of file > fs/xfs/linux-2.6/xfs_buf.c. Return address = 0xffffffffa02a6213 > > > server2: > Dec 27 00:30:02 localhost kernel: XFS (sdb): xfs_iunlink_remove: > xfs_inotobp() returned error 22. > Dec 27 00:30:02 localhost kernel: XFS (sdb): xfs_inactive: xfs_ifree > returned error 22 > Dec 27 00:30:02 localhost kernel: XFS (sdb): > xfs_do_force_shutdown(0x1) called from line 1184 of file > fs/xfs/xfs_vnodeops.c. Return address = 0xffffffffa022420a > Dec 27 00:30:02 localhost kernel: XFS (sdb): xfs_imap_to_bp: > xfs_trans_read_buf() returned error 5. > Dec 27 00:30:02 localhost kernel: XFS (sdb): I/O Error Detected. > Shutting down filesystem > Dec 27 00:30:02 localhost kernel: XFS (sdb): Please umount the > filesystem and rectify the problem(s) > Dec 27 00:30:27 localhost kernel: XFS (sdb): xfs_log_force: error 5 returned. > Dec 27 00:30:57 localhost kernel: XFS (sdb): xfs_log_force: error 5 returned. > Dec 27 00:31:27 localhost kernel: XFS (sdb): xfs_log_force: error 5 returned. > Dec 27 00:31:57 localhost kernel: XFS (sdb): xfs_log_force: error 5 returned. > Dec 27 00:32:27 localhost kernel: XFS (sdb): xfs_log_force: error 5 returned. > Dec 27 00:32:57 localhost kernel: XFS (sdb): xfs_log_force: error 5 returned. > Dec 27 00:33:27 localhost kernel: XFS (sdb): xfs_log_force: error 5 returned. > Dec 27 00:33:57 localhost kernel: XFS (sdb): xfs_log_force: error 5 returned. > Dec 27 00:34:27 localhost kernel: XFS (sdb): xfs_log_force: error 5 returned. > Dec 27 00:34:57 localhost kernel: XFS (sdb): xfs_log_force: error 5 returned. > Dec 27 00:35:27 localhost kernel: XFS (sdb): xfs_log_force: error 5 returned. > Dec 27 00:35:32 localhost smbd[25481]: [2012/12/27 00:35:32.424643, 0] > printing/print_cups.c:109(cups_connect) > Dec 27 00:35:32 localhost smbd[25481]: Unable to connect to CUPS > server localhost:631 - Connection refused > Dec 27 00:35:32 localhost smbd[2061]: [2012/12/27 00:35:32.424906, 0] > printing/print_cups.c:468(cups_async_callback) > Dec 27 00:35:32 localhost smbd[2061]: failed to retrieve printer list: > NT_STATUS_UNSUCCESSFUL > Dec 27 00:35:57 localhost kernel: XFS (sdb): xfs_log_force: error 5 returned. > Dec 27 00:36:27 localhost kernel: XFS (sdb): xfs_log_force: error 5 returned. > Dec 27 00:36:57 localhost kernel: XFS (sdb): xfs_log_force: error 5 returned. > > _______________________________________________ xfs mailing list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
