(against latest gfs in cvs)
scenario:
- 3 machines in cluster
- one importing gnbd, two directly mounted to shared fc raid
* all 3 performing io
* reboot one machine, all of the machines hung on io
attempted to get rebooted machine to join the cluster, one machine spat
out the following:
kernel BUG at /usr/src/GFS/cluster/cman-kernel/src/membership.c:611!
invalid operand: 0000 [#1]
SMP
Modules linked in: gnbd gfs lock_dlm dlm cman lock_harness 8250
serial_core dm_mod
CPU: 0
EIP: 0060:[<f88efc18>] Not tainted
EFLAGS: 00010246 (2.6.8.1)
EIP is at send_joinconf+0x10/0x75 [cman]
eax: 00000000 ebx: 00000003 ecx: 018a60be edx: c180e08c
esi: f7ff6fc0 edi: 00000000 ebp: 00000001 esp: f324fe74
ds: 007b es: 007b ss: 0068
Process cman_memb (pid: 822, threadinfo=f324e000 task=f3242c70)
Stack: f8907380 00000000 00000003 f7ff6fc0 00000000 00000003 f88f15ec
f8907380
018a7446 c01193bf c0331140 c0401a8d f7ff6fc0 0000824d 0000824d
0000824d
00000000 f7ff6fc0 00000000 00000007 f88f087f 00000000 00000000
00000038
Call Trace:
[<f88f15ec>] do_process_startack+0x14d/0x38e [cman]
[<c01193bf>] __call_console_drivers+0x55/0x57
[<f88f087f>] start_transition+0x20f/0x2c1 [cman]
[<f88ccee1>] cman_callback+0x35/0x38 [dlm]
[<f88e952a>] notify_kernel_listeners+0x41/0x68 [cman]
[<f88f0a94>] a_node_just_died+0x163/0x181 [cman]
[<f88f2a5c>] do_process_leave+0x6b/0x7d [cman]
[<f88f043c>] do_membership_packet+0x98/0x1f0 [cman]
[<f88f2e9c>] dispatch_messages+0xe3/0x104 [cman]
[<f88ef492>] membership_kthread+0x216/0x3e6 [cman]
[<c0103d22>] ret_from_fork+0x6/0x14
[<c01157c8>] default_wake_function+0x0/0x12
[<f88ef27c>] membership_kthread+0x0/0x3e6 [cman]
[<c0102161>] kernel_thread_helper+0x5/0xb
Code: 0f 0b 63 02 40 bd 8f f8 89 44 24 10 c7 05 f0 73 90 f8 02 00
kernel BUG at /usr/src/GFS/cluster/cman-kernel/src/membership.c:611!
invalid operand: 0000 [#1]
CPU: 0
EIP: 0060:[<f88efc18>] Not tainted
Using defaults from ksymoops -t elf32-i386 -a i386
EFLAGS: 00010246 (2.6.8.1)
eax: 00000000 ebx: 00000003 ecx: 018a60be edx: c180e08c
esi: f7ff6fc0 edi: 00000000 ebp: 00000001 esp: f324fe74
ds: 007b es: 007b ss: 0068
Stack: f8907380 00000000 00000003 f7ff6fc0 00000000 00000003 f88f15ec
f8907380
018a7446 c01193bf c0331140 c0401a8d f7ff6fc0 0000824d 0000824d
0000824d
00000000 f7ff6fc0 00000000 00000007 f88f087f 00000000 00000000
00000038
[<f88f15ec>] do_process_startack+0x14d/0x38e [cman]
[<c01193bf>] __call_console_drivers+0x55/0x57
[<f88f087f>] start_transition+0x20f/0x2c1 [cman]
[<f88ccee1>] cman_callback+0x35/0x38 [dlm]
[<f88e952a>] notify_kernel_listeners+0x41/0x68 [cman]
[<f88f0a94>] a_node_just_died+0x163/0x181 [cman]
[<f88f2a5c>] do_process_leave+0x6b/0x7d [cman]
[<f88f043c>] do_membership_packet+0x98/0x1f0 [cman]
[<f88f2e9c>] dispatch_messages+0xe3/0x104 [cman]
[<f88ef492>] membership_kthread+0x216/0x3e6 [cman]
[<c0103d22>] ret_from_fork+0x6/0x14
[<c01157c8>] default_wake_function+0x0/0x12
[<f88ef27c>] membership_kthread+0x0/0x3e6 [cman]
[<c0102161>] kernel_thread_helper+0x5/0xb
Code: 0f 0b 63 02 40 bd 8f f8 89 44 24 10 c7 05 f0 73 90 f8 02 00
>>EIP; f88efc18 <pg0+384c4c18/3fbd3000> <=====
>>ecx; 018a60be Before first symbol
>>edx; c180e08c <pg0+13e308c/3fbd3000>
>>esi; f7ff6fc0 <pg0+37bcbfc0/3fbd3000>
>>esp; f324fe74 <pg0+32e24e74/3fbd3000>
Code; f88efc18 <pg0+384c4c18/3fbd3000>
00000000 <_EIP>:
Code; f88efc18 <pg0+384c4c18/3fbd3000> <=====
0: 0f 0b ud2a <=====
Code; f88efc1a <pg0+384c4c1a/3fbd3000>
2: 63 02 arpl %ax,(%edx)
Code; f88efc1c <pg0+384c4c1c/3fbd3000>
4: 40 inc %eax
Code; f88efc1d <pg0+384c4c1d/3fbd3000>
5: bd 8f f8 89 44 mov $0x4489f88f,%ebp
Code; f88efc22 <pg0+384c4c22/3fbd3000>
a: 24 10 and $0x10,%al
Code; f88efc24 <pg0+384c4c24/3fbd3000>
c: c7 05 f0 73 90 f8 02 movl $0x2,0xf89073f0
Code; f88efc2b <pg0+384c4c2b/3fbd3000>
13: 00 00 00
