accessing files on gfs caused fatal error, gfs on both boxes crumbles..

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



The gfs share was responding fine for the most part. Not sure if this is related to the postfix question I asked, but a I was moving files back and forth manually (just getting the boxes setup running the same things and configured the same way), I placed something on the share from server 2 (goumang), then when i tried to access it via box 1(rushou) things went bad.

I had placed the source for one of the apps that I am going to install (rather small thing) on the share, ran ./configure and voila..

[root@rushou rabl_client-1.0.0]# ./configure \
> --prefix=/ \
> --exec-prefix=/usr \
> --includedir=/usr/include
-bash: ./configure: Permission denied
[root@rushou rabl_client-1.0.0]#
[root@rushou rabl_client-1.0.0]# ls -l
ls: .: Input/output error
[root@rushou rabl_client-1.0.0]# ls -lZ
ls: .: Input/output error

I tried restarting gfs2 on rushou, and it worked once, went back into the directory, ran it again (hey, gotta replicate it right?) and same thing, except on restart i got this:
[root@rushou mnt]# service gfs2 restart
Unmounting GFS2 filesystems: [ OK ]
FATAL: Module lock_dlm is in use.
FATAL: Module gfs2 is in use.
Mounting GFS2 filesystems: /sbin/mount.gfs2: error: jid is -2, try rw
/sbin/mount.gfs2: error mounting lockproto lock_dlm
[FAILED]

Then when i tried to do anything on the gfs mount on goumang... LOCK.

So maybe running a configure from the gfs drive wasnt so bright... but in theory with the load i plan on hitting this thing with (mail server) I would hope a small configure setup wouldnt cause to much issue.

i've attached the info from /var/log/messages

[root@rushou /]# uname -a
Linux rushou.sgc 2.6.18-1.2849.fc6 #1 SMP Fri Nov 10 12:36:14 EST 2006 i686 i686 i386 GNU/Linux
[root@rushou /]# rpm -qa | grep gfs
gfs2-utils-0.1.7-1.fc6
[root@rushou /]# rpm -qa | grep cman
cman-2.0.18-2.fc6
[root@rushou /]# rpm -qa | grep lvm
lvm2-2.02.06-4
lvm2-cluster-2.02.06-1.5
system-config-lvm-1.0.18-1.2.FC6


any other information that would be useful?

-greg

--
http://www.gvtc.com
--
“While it is possible to change without improving, it is impossible to improve without changing.” -anonymous

“only he who attempts the absurd can achieve the impossible.” -anonymous

Nov 28 23:32:14 rushou kernel: GFS2: fsid=outMail:data.0: fatal: filesystem consistency error
Nov 28 23:32:14 rushou kernel: GFS2: fsid=outMail:data.0:   inode = 1086093 3284792
Nov 28 23:32:14 rushou kernel: GFS2: fsid=outMail:data.0:   function = gfs2_drevalidate, file = fs/gfs2/ops_dentry.c, line = 80
Nov 28 23:32:14 rushou kernel: GFS2: fsid=outMail:data.0: about to withdraw from the cluster
Nov 28 23:32:14 rushou kernel: GFS2: fsid=outMail:data.0: waiting for outstanding I/O
Nov 28 23:32:14 rushou kernel: GFS2: fsid=outMail:data.0: telling LM to withdraw
Nov 28 23:32:14 rushou kernel: GFS2: fsid=outMail:data.0: withdrawn
Nov 28 23:32:14 rushou kernel:  [<c0405876>] dump_trace+0x69/0x1af
Nov 28 23:32:14 rushou kernel:  [<c04059d4>] show_trace_log_lvl+0x18/0x2c
Nov 28 23:32:14 rushou kernel:  [<c0405f83>] show_trace+0xf/0x11
Nov 28 23:32:14 rushou kernel:  [<c0406080>] dump_stack+0x15/0x17
Nov 28 23:32:14 rushou kernel:  [<f8e88063>] gfs2_lm_withdraw+0x9a/0xa5 [gfs2]
Nov 28 23:32:14 rushou kernel:  [<f8e9a380>] gfs2_consist_inode_i+0x6f/0x75 [gfs2]
Nov 28 23:32:14 rushou kernel:  [<f8e8de87>] gfs2_drevalidate+0x13e/0x1ef [gfs2]
Nov 28 23:32:14 rushou kernel:  [<c0476e90>] do_lookup+0x114/0x15a
Nov 28 23:32:14 rushou kernel:  [<c0478c37>] __link_path_walk+0x8ad/0xd73
Nov 28 23:32:14 rushou kernel:  [<c0479146>] link_path_walk+0x49/0xbd
Nov 28 23:32:14 rushou kernel:  [<c0479523>] do_path_lookup+0x21a/0x26b
Nov 28 23:32:14 rushou kernel:  [<c0479ce8>] __user_walk_fd+0x2f/0x40
Nov 28 23:32:14 rushou kernel:  [<c0473803>] vfs_stat_fd+0x19/0x40
Nov 28 23:32:14 rushou kernel:  [<c04738b7>] sys_stat64+0xf/0x23
Nov 28 23:32:14 rushou kernel:  [<c0404027>] syscall_call+0x7/0xb
Nov 28 23:32:14 rushou kernel: DWARF2 unwinder stuck at syscall_call+0x7/0xb
Nov 28 23:32:14 rushou kernel: Leftover inexact backtrace:
Nov 28 23:32:14 rushou kernel:  =======================
Nov 28 23:33:29 rushou multipathd: dm-1: umount map (uevent)
Nov 28 23:33:34 rushou kernel: GFS2: fsid=: Trying to join cluster "lock_dlm", "outMail:data"
Nov 28 23:33:34 rushou kernel: dlm: data: recover 1
Nov 28 23:33:34 rushou kernel: GFS2: fsid=outMail:data.0: Joined cluster. Now mounting FS...
Nov 28 23:33:34 rushou kernel: dlm: data: add member 1
Nov 28 23:33:34 rushou kernel: dlm: data: add member 2
Nov 28 23:33:34 rushou kernel: dlm: data: total members 2 error 0
Nov 28 23:33:34 rushou kernel: dlm: data: dlm_recover_directory
Nov 28 23:33:34 rushou kernel: dlm: data: dlm_recover_directory 18762 entries
Nov 28 23:33:34 rushou kernel: dlm: data: recover 1 done: 97 ms
Nov 28 23:33:34 rushou kernel: GFS2: fsid=outMail:data.0: jid=0, already locked for use
Nov 28 23:33:34 rushou kernel: GFS2: fsid=outMail:data.0: jid=0: Looking at journal...
Nov 28 23:33:34 rushou kernel: GFS2: fsid=outMail:data.0: jid=0: Done
Nov 28 23:33:54 rushou kernel: GFS2: fsid=outMail:data.0: fatal: filesystem consistency error
Nov 28 23:33:54 rushou kernel: GFS2: fsid=outMail:data.0:   inode = 1086093 3284792
Nov 28 23:33:54 rushou kernel: GFS2: fsid=outMail:data.0:   function = gfs2_drevalidate, file = fs/gfs2/ops_dentry.c, line = 80
Nov 28 23:33:54 rushou kernel: GFS2: fsid=outMail:data.0: about to withdraw from the cluster
Nov 28 23:33:54 rushou kernel: GFS2: fsid=outMail:data.0: waiting for outstanding I/O
Nov 28 23:33:54 rushou kernel: GFS2: fsid=outMail:data.0: telling LM to withdraw
Nov 28 23:33:54 rushou kernel: GFS2: fsid=outMail:data.0: withdrawn
Nov 28 23:33:54 rushou kernel:  [<c0405876>] dump_trace+0x69/0x1af
Nov 28 23:33:54 rushou kernel:  [<c04059d4>] show_trace_log_lvl+0x18/0x2c
Nov 28 23:33:54 rushou kernel:  [<c0405f83>] show_trace+0xf/0x11
Nov 28 23:33:54 rushou kernel:  [<c0406080>] dump_stack+0x15/0x17
Nov 28 23:33:54 rushou kernel:  [<f8e88063>] gfs2_lm_withdraw+0x9a/0xa5 [gfs2]
Nov 28 23:33:54 rushou kernel:  [<f8e9a380>] gfs2_consist_inode_i+0x6f/0x75 [gfs2]
Nov 28 23:33:54 rushou kernel:  [<f8e8de87>] gfs2_drevalidate+0x13e/0x1ef [gfs2]
Nov 28 23:33:54 rushou kernel:  [<c0476e90>] do_lookup+0x114/0x15a
Nov 28 23:33:54 rushou kernel:  [<c0478c37>] __link_path_walk+0x8ad/0xd73
Nov 28 23:33:54 rushou kernel:  [<c0479146>] link_path_walk+0x49/0xbd
Nov 28 23:33:54 rushou kernel:  [<c0479523>] do_path_lookup+0x21a/0x26b
Nov 28 23:33:54 rushou kernel:  [<c0479ce8>] __user_walk_fd+0x2f/0x40
Nov 28 23:33:54 rushou kernel:  [<c0473803>] vfs_stat_fd+0x19/0x40
Nov 28 23:33:54 rushou kernel:  [<c04738b7>] sys_stat64+0xf/0x23
Nov 28 23:33:54 rushou kernel:  [<c0404027>] syscall_call+0x7/0xb
Nov 28 23:33:54 rushou kernel: DWARF2 unwinder stuck at syscall_call+0x7/0xb
Nov 28 23:33:54 rushou kernel: Leftover inexact backtrace:
Nov 28 23:33:54 rushou kernel:  =======================
Nov 28 23:34:18 rushou multipathd: dm-1: umount map (uevent)
Nov 28 23:32:14 goumang kernel: GFS2: fsid=outMail:data.1: jid=0: Trying to acquire journal lock...
Nov 28 23:32:14 goumang kernel: dlm: data: recover 3
Nov 28 23:32:14 goumang kernel: dlm: data: remove member 2
Nov 28 23:32:14 goumang kernel: dlm: data: total members 1 error 0
Nov 28 23:32:14 goumang kernel: dlm: data: dlm_recover_directory
Nov 28 23:32:14 goumang kernel: dlm: data: dlm_recover_directory 37412 entries
Nov 28 23:32:14 goumang kernel: dlm: data: pre recover waiter lkid 1480322 type 11 flags 0
Nov 28 23:32:14 goumang kernel: dlm: data: dlm_purge_locks
Nov 28 23:32:14 goumang kernel: dlm: data: dlm_recover_masters
Nov 28 23:32:14 goumang kernel: dlm: data: dlm_recover_masters 44 resources
Nov 28 23:32:14 goumang kernel: dlm: data: dlm_recover_locks
Nov 28 23:32:14 goumang kernel: dlm: data: dlm_recover_locks 0 locks
Nov 28 23:32:14 goumang kernel: dlm: data: dlm_recover_rsbs
Nov 28 23:32:14 goumang kernel: dlm: data: dlm_recover_rsbs 37456 rsbs
Nov 28 23:32:14 goumang kernel: dlm: data: recover_waiters_post 1480322 type 11 flags 0        9               0
Nov 28 23:32:14 goumang kernel: GFS2: fsid=outMail:data.1: jid=0: Looking at journal...
Nov 28 23:32:14 goumang kernel: dlm: data: recover 3 done: 186 ms
Nov 28 23:32:14 goumang kernel: GFS2: fsid=outMail:data.1: jid=0: Done
Nov 28 23:33:34 goumang kernel: dlm: data: recover 5
Nov 28 23:33:34 goumang kernel: dlm: data: add member 2
Nov 28 23:33:34 goumang kernel: dlm: data: total members 2 error 0
Nov 28 23:33:34 goumang kernel: dlm: data: dlm_recover_directory
Nov 28 23:33:34 goumang kernel: dlm: data: dlm_recover_directory 18638 entries
Nov 28 23:33:34 goumang kernel: dlm: data: recover 5 done: 105 ms
Nov 28 23:33:54 goumang kernel: GFS2: fsid=outMail:data.1: jid=0: Trying to acquire journal lock...
Nov 28 23:33:54 goumang kernel: GFS2: fsid=outMail:data.1: jid=0: Busy
Nov 28 23:33:54 goumang kernel: dlm: data: recover 7
Nov 28 23:33:54 goumang kernel: dlm: data: remove member 2
Nov 28 23:33:54 goumang kernel: dlm: data: total members 1 error 0
Nov 28 23:33:54 goumang kernel: dlm: data: dlm_recover_directory
Nov 28 23:33:54 goumang kernel: dlm: data: dlm_recover_directory 37400 entries
Nov 28 23:33:54 goumang kernel: dlm: data: dlm_purge_locks
Nov 28 23:33:54 goumang kernel: dlm: data: dlm_recover_masters
Nov 28 23:33:54 goumang kernel: dlm: data: dlm_recover_masters 0 resources
Nov 28 23:33:54 goumang kernel: dlm: data: dlm_recover_locks
Nov 28 23:33:54 goumang kernel: dlm: data: dlm_recover_locks 0 locks
Nov 28 23:33:54 goumang kernel: dlm: data: dlm_recover_rsbs
Nov 28 23:33:54 goumang kernel: dlm: data: dlm_recover_rsbs 37400 rsbs
Nov 28 23:33:54 goumang kernel: dlm: data: recover 7 done: 103 ms
--
Linux-cluster mailing list
Linux-cluster@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/linux-cluster

[Index of Archives]     [Corosync Cluster Engine]     [GFS]     [Linux Virtualization]     [Centos Virtualization]     [Centos]     [Linux RAID]     [Fedora Users]     [Fedora SELinux]     [Big List of Linux Books]     [Yosemite Camping]

  Powered by Linux