hi,
after we switch our servers from centos-3 to centos-4 (aka. rhel-4) one
of our server always crash once a week without any oops. this happneds
with both the normal kernel-2.6.9-11.EL and
kernel-2.6.9-11.106.unsupported. after we change the motherboard, the
raid contorller and the cables too we still got it. finally we start
netdump and last but not least yesterday we got a crash log and a core
file. it seems there is a bug in the raid5 code of the kernel.
this is our backup server with 8 x 200GB hdd in a raid5 (for the data)
plus 2 x 40GB hdd in raid1 (for the system) with 3ware 8xxx raid
contorller, running. i attached the netdump log of the last crash.
how can i fix it?
yours.
--
Levente "Si vis pacem para bellum!"
RAID5 conf printout:
--- rd:8 wd:8 fd:0
disk 0, o:1, dev:sda1
disk 1, o:1, dev:sdb1
disk 2, o:1, dev:sdc1
disk 3, o:1, dev:sdd1
disk 4, o:1, dev:sde1
disk 5, o:1, dev:sdf1
disk 6, o:1, dev:sdg1
disk 7, o:1, dev:sdh1
Unable to handle kernel NULL pointer dereference at virtual address 00000000
printing eip:
00000000
*pde = 0f94a067
Oops: 0000 [#1]
Modules linked in: cifs nls_utf8 ncpfs nfsd exportfs lockd sunrpc parport_pc lp parport netconsole netdump i2c_dev i2c_core ipx dm_mod e1000 tg3 floppy ext3 jbd raid5 xor raid1 3w_xxxx sd_mod scsi_mod
CPU: 0
EIP: 0060:[<00000000>] Not tainted VLI
EFLAGS: 00010246 (2.6.9-11.106.unsupported)
EIP is at 0x0
eax: c1806138 ebx: c018961c ecx: 00000016 edx: c035c7f4
esi: e7182200 edi: 00000001 ebp: c18fb380 esp: f7878f34
ds: 007b es: 007b ss: 0068
Process md2_raid5 (pid: 224, threadinfo=f7878000 task=f7872600)
Stack: f7b973c0 f8879a26 md_thread+0x20d/0x23a
[<c011ceaf>] autoremove_wake_function+0x0/0x2d
[<c030ce1a>] ret_from_fork+0x6/0x14
[<c011ceaf>] autoremove_wake_function+0x0/0x2d
[<c02a183f>] md_thread+0x0/0x23a
[<c01041d9>] kernel_thread_helper+0x5/0xb
Code: Bad EIP value.
Pid: 224, comm: md2_raid5
EIP: 0060:[<00000000>] CPU: 0
EIP is at 0x0
EFLAGS: 00010246 Not tainted (2.6.9-11.106.unsupported)
EAX: c1806138 EBX: c018961c ECX: 00000016 EDX: c035c7f4
ESI: e7182200 EDI: 00000001 EBP: c18fb380 DS: 007b ES: 007b
CR0: 8005003b CR2: ffffffd5 CR3: 0fd6b000 CR4: 000006d0
[<f8879a26>] handle_stripe+0xfca/0x1207 [raid5]
[<f887a7d5>] raid5d+0x197/0x2ab [raid5]
[<c02a1a4c>] md_thread+0x20d/0x23a
[<c011ceaf>] autoremove_wake_function+0x0/0x2d
[<c030ce1a>] ret_from_fork+0x6/0x14
[<c011ceaf>] autoremove_wake_function+0x0/0x2d
[<c02a183f>] md_thread+0x0/0x23a
[<c01041d9>] kernel_thread_helper+0x5/0xb
sibling
task PC pid father child younger older
init S C01458E9 920 1 0 2 (NOTLB)
f7f44eb0 00000086 00000055 xfrm_state_flush+0x2/0x289 tcp_poll+0x31/0x144
[<c01768e1>] do_select+0x347/0x378
[<c0176461>] __pollwait+0x0/0x94
[<c0176c05>] sys_select+0x2e0/0x43a
[<c030cefb>] syscall_call+0x7/0xb
ntpd S 000000D0 2516 2196 1 2219 2172 (NOTLB)
f0885eb0 00000082 00000246 000000d0 cf8553a0 000021cd 5197434f 00003abd
f697d2a0 f697d42c f6a0d580 7fffffff 00000000 f0885f74 c030b7e5 f69a5980
00000000 f0885f58 f69a5980 f106ed18 c017648e 00000246 f106b800 f0885f58
Call Trace:
[<c030b7e5>] schedule_timeout+0x50/0x10c
[<c017648e>] __pollwait+0x2d/0x94
[<c02aeeac>] datagram_poll+0x25/0xd1
[<c01768e1>] do_select+0x347/0x378
[<c0176461>] __pollwait+0x0/0x94
[<c0176c05>] sys_select+0x2e0/0x43a
[<c01058d8>] sys_sigreturn+0x1ce/0x1f2
[<c030cefb>] syscall_call+0x7/0xb
rpc.rquotad S 00000000 3416 2219 1 2223 2196 (NOTLB)
f65a4f1c 00000082 00000001 00000000 f697ccd0 00030ee2 966e2a43 00000033
f69b1320 f69b14ac 00000000 7fffffff f65a4fa0 f0888ba0 c030b7e5 f106b580
f65a4fa0 f106e518 c02aeeac f6a7a780 c03806c0 00000145 f0888bb0 00000001
Call Trace:
[<c030b7e5>] schedule_timeout+0x50/0x10c
[<c02aeeac>] datagram_poll+0x25/0xd1
[<c02a8f2c>] sock_poll+0x12/0x14
[<c0176db3>] do_pollfd+0x54/0x77
[<c0176e63>] do_poll+0x8d/0xab
[<c0177020>] sys_poll+0x19f/0x24f
[<c0176461>] __pollwait+0x0/0x94
[<c030cefb>] syscall_call+0x7/0xb
nfsd S FF4DCFB0 2316 2223 1 2224 2219 (L-TLB)
f05fff10 00000046 00000002 ff4dcfb0 f69c4dd0 000013cc c85b7bd5 000037c8
f69b0d50 f69b0edc 03db8cae 03db8cae 0000000b c1993c00 c030b886 f5675f18
c035b0d0 03db8cae 1d244b3c 00000000 00000005 c031a0b5 c031c25c 000000a8
Call Trace:
[<c030b886>] schedule_timeout+0xf1/0x10c
[<c0129336>] process_timeout+0x0/0x5
[<f8ade6bb>] svc_recv+0x325/0x65b [sunrpc]
[<c011b856>] default_wake_function+0x0/0xc
[<c011b921>] __wake_up+0x6e/0xca
[<c011b856>] default_wake_function+0x0/0xc
[<c012d587>] sigprocmask+0x140/0x1f4
[<f8b2e44d>] nfsd+0x1ae/0x540 [nfsd]
[<f8b2e29f>] nfsd+0x0/0x540 [nfsd]
[<c01041d9>] kernel_thread_helper+0x5/0xb
nfsd S 000037C3 3472 2224 1 2225 2223 (L-TLB)
f1513f10 00000046 431bf158 000037c3 f78731a0 00001630 431bf158 000037c3
f6a21870 f6a219fc 03db2fe5 03db2fe5 0000000b f7f58400 c030b886 f1806f18
c035b0c0 03db2fe5 1d244b3c 00000000 00000005 c031a0b5 c031c25c 000000a8
Call Trace:
[<c030b886>] schedule_timeout+0xf1/0x10c
[<c0129336>] process_timeout+0x0/0x5 schedule_timeout+0xf1/0x10c
[<c0129336>] process_timeout+0x0/0x5
[<f8ade6bb>] svc_recv+0x325/0x65b [sunrpc]
[<c011b856>] default_wake_function+0x0/0xc
[<c011b856>] default_wake_function+0x0/0xc
[<c012d587>] sigprocmask+0x140/0x1f4
[<f8b2e44d>] nfsd+0x1ae/0x540 [nfsd]
[<f8b2e29f>] nfsd+0x0/0x540 [nfsd]
kernel_thread_helper+0x5/0xb schedule_timeout+0x50/0x10c
[<c02ae7c6>] skb_recv_datagram+0x15a/0x18f
[<f8ade6bb>] svc_recv+0x325/0x65b [sunrpc]
[<c011b856>] default_wake_function+0x0/0xc
[<c01041d4>] kernel_thread_helper+0x0/0xb
[<c011b856>] default_wake_function+0x0/0xc
[<f894c6f7>] lockd+0x1ae/0x33f [lockd]
[<f894c549>] lockd+0x0/0x33f [lockd]
[<c01041d9>] kernel_thread_helper+0x5/0xb
rpciod S C0129BCC 3752 2231 1 2232 2230 (L-TLB)
f5665f9c 00000046 f5665000 c0129bcc f69b0780 00000a63 99004822 00000033
f69c4dd0 f69c4f5c f5665000 00000000 f5665fb8 00000000 f8ada30b 00000000
f69c4dd0 c011ceaf f5665fd0 f5665fd0 f05fff10 c030ce1a f69b0d50 00000000
free_uid+0x15/0xac
[<f8ada30b>] rpciod+0x556/0x74b [sunrpc]
[<c011ceaf>] autoremove_wake_function+0x0/0x2d
[<c030ce1a>] ret_from_fork+0x6/0x14
[<c011ceaf>] autoremove_wake_function+0x0/0x2d
[<f8ad9db5>] rpciod+0x0/0x74b [sunrpc]
[<c01041d9>] kernel_thread_helper+0x5/0xb
nfsd S 00000000 2136 2232 1 2236 2231 (L-TLB)
f5675f10 00000046 f7f59864 00000000 f7f59864 00000886 cf330c8a 000037c8
f69c4800 f69c498c 03db8d21 03db8d21 0000000b f7f59800 c030b886 c035b0d0
f05fff18 03db8d21 1d244b3c 00000000 00000005 c031a0b5 c031c25c 000000a8
Call Trace:
[<c030b886>] schedule_timeout+0xf1/0x10c
[<c0129336>] process_timeout+0x0/0x5
[<f8ade6bb>] svc_recv+0x325/0x65b [sunrpc]
[<c011b856>] default_wake_function+0x0/0xc
[<f8adc82b>] svc_process+0x480/0x6cd [sunrpc]
[<c011b856>] default_wake_function+0x0/0xc
[<c012d587>] sigprocmask+0x140/0x1f4
[<f8b2e44d>] nfsd+0x1ae/0x540 [nfsd]
[<f8b2e29f>] nfsd+0x0/0x540 [nfsd]
[<c01041d9>] kernel_thread_helper+0x5/0xb
rpc.mountd S C16A7000 3416 2236 1 2256 2232 (NOTLB)
f567eeb0 00000086 c01458e9 c16a7000 f697ccd0 0004c4da 9d1f8094 00000033
f69c5970 f69c5afc f6a0f480 7fffffff 00000000 f567ef74 c030b7e5 00000000
00df9100 00000246 f5664580 f567ef58 f1818598 c02d1fed f5663d00 f567ef58
Call Trace:
buffered_rmqueue+0x1c4/0x1e7
[<c030b7e5>] schedule_timeout+0x50/0x10c
[<c02d1fed>] tcp_poll+0x31/0x144
[<c02aeeac>] datagram_poll+0x25/0xd1
[<c01768e1>] do_select+0x347/0x378
[<c0176461>] __pollwait+0x0/0x94
[<c0176c05>] sys_select+0x2e0/0x43a
[<c030b650>] __cond_resched+0x14/0x3b
[<c030cefb>] syscall_call+0x7/0xb
rpc.idmapd S 00003ABB 3288 2256 1 2316 2236 (NOTLB)
f088af1c 00000086 6792e02d 00003abb cf919770 00000f18 912b8daa 00003abc
f6a6d1a0 f6a6d32c 03d65077 03d65077 00000000 00001388 c030b886 f6bc7eb8
cf9a0da4 03d65077 1d244b3c 00000000 00000005 c031a0b5 c031c25c 000000a8
Call Trace:
[<c030b886>] schedule_timeout+0xf1/0x10c
[<c0129336>] process_timeout+0x0/0x5
[<c018fd78>] ep_poll+0xb7/0x11e
[<c011b856>] default_wake_function+0x0/0xc
[<c018f041>] sys_epoll_wait+0x70/0x83
[<c030cefb>] syscall_call+0x7/0xb
[<c030007b>] xfrm_state_flush+0x2/0x289
master S C01458E9 2288 2316 1 2325 2331 2256 (NOTLB)
f65d4eb0 00000086 00000055 c01458e9 f08e80b0 000024bb 32f4d56a 00003ab7
f697ccd0 f697ce5c 03d6cd16 03d6cd16 00000000 f65d4f74 c030b886 c0439020
f04d0f54 03d6cd16 1d244b3c 00000000 00000005 c031a0b5 c031c25c 000000a8
Call Trace:
[<c01458e9>] buffered_rmqueue+0x1c4/0x1e7
[<c030b886>] schedule_timeout+0xf1/0x10c
[<c0129336>] process_timeout+0x0/0x5
[<c01768e1>] do_select+0x347/0x378
[<c0176461>] __pollwait+0x0/0x94
[<c0176c05>] sys_select+0x2e0/0x43a
[<c01243db>] sys_time+0xf/0x58
[<c030cefb>] syscall_call+0x7/0xb
qmgr S 00003AA6 2368 2325 2316 5271 (NOTLB)
f65d9eb0 00000086 aff835d3 00003aa6 f780a1b0 0000586b aff87065 00003aa6
f6a6d770 f6a6d8fc 03de71f1 03de71f1 00000000 f65d9f74 c030b886 c035b128
c035b128 03de71f1 1d244b3c 00000000 00000005 c031a0b5 c031c25c 000000a8
Call Trace:
[<c030b886>] schedule_timeout+0xf1/0x10c
[<c0129336>] process_timeout+0x0/0x5
[<c01768e1>] do_select+0x347/0x378
[<c0176461>] __pollwait+0x0/0x94
[<c0176c05>] sys_select+0x2e0/0x43a
[<c030b650>] __cond_resched+0x14/0x3b
[<c01243db>] sys_time+0xf/0x58
[<c030cefb>] syscall_call+0x7/0xb
[<c030007b>] xfrm_state_flush+0x2/0x289
gpm S 00000034 2764 2331 1 2340 2316 (NOTLB)
f04cceb0 00000082 30984ef4 00000034 f780a1b0 00031940 309a3434 00000034
f78a60b0 f78a623c 0524d0d9 0524d0d9 00000000 f04ccf74 c030b886 c035b368
c035b368 0524d0d9 1d244b3c 00000000 00000005 c031a0b5 c031c25c 000000a8
Call Trace:
[<c030b886>] schedule_timeout+0xf1/0x10c
[<c0129336>] process_timeout+0x0/0x5
[<c01768e1>] do_select+0x347/0x378
[<c0176461>] __pollwait+0x0/0x94
[<c0176c05>] sys_select+0x2e0/0x43a
[<c02aa01a>] sys_socketcall+0xde/0x1dd
[<c01244b5>] sys_gettimeofday+0x53/0xac
[<c030cefb>] syscall_call+0x7/0xb
crond S 000000FF 2616 2340 1 5980 2349 2331 (NOTLB)
f65dcf4c 00000082 00000000 000000ff cfdc0dd0 000025c3 16273dd1 00003abc
f0ba3970 f0ba3afc 03d71f46 03d71f46 0000ea6b fa09bac0 c030b886 c035b040
f8b57e98 03d71f46 1d244b3c 00000000 00000005 c031a0b5 c031c25c 000000a8
Call Trace:
[<c030b886>] schedule_timeout+0xf1/0x10c
[<c0129336>] process_timeout+0x0/0x5
[<c012944e>] sys_nanosleep+0x103/0x1a1
[<c030cefb>] syscall_call+0x7/0xb
smbd S C035CA9C 1752 2349 1 2374 2353 2340 (NOTLB)
f04caeb0 00000086 000000d0 c035ca9c c01459c0 0001f6f4 b600904c 00003133
f0ba33a0 f0ba352c f04d1b80 7fffffff 00000000 f04caf74 c030b7e5 f0bbac98
c017648e 00000246 f0b43580 f04caf58 f0bba898 c02d1fed 00000000 00000000
Call Trace:
[<c01459c0>] __alloc_pages+0xb4/0x29d
[<c030b7e5>] schedule_timeout+0x50/0x10c
[<c017648e>] __pollwait+0x2d/0x94
[<c02d1fed>] tcp_poll+0x31/0x144
[<c01768e1>] do_select+0x347/0x378
[<c0176461>] __pollwait+0x0/0x94
[<c0176c05>] sys_select+0x2e0/0x43a
[<c01058d8>] sys_sigreturn+0x1ce/0x1f2
[<c030cefb>] syscall_call+0x7/0xb
nmbd S 00003ABB 2320 2353 1 2370 2349 (NOTLB)
f04d3eb0 00000086 78a3db2d 00003abb cffc18f0 0001365b 78ab8b56 00003abb
f0ba2dd0 f0ba2f5c 03d65194 03d65194 00000000 f04d3f74 c030b886 d11d6da4
c035ade8 03d65194 1d244b3c 00000000 00000005 c031a0b5 c031c25c 000000a8
Call Trace:
[<c030b886>] schedule_timeout+0xf1/0x10c
[<c0129336>] process_timeout+0x0/0x5
[<c01768e1>] do_select+0x347/0x378
[<c0176461>] __pollwait+0x0/0x94
[<c0176c05>] sys_select+0x2e0/0x43a
[<c012d587>] sigprocmask+0x140/0x1f4
[<c030cefb>] syscall_call+0x7/0xb
[<c030007b>] xfrm_state_flush+0x2/0x289
atd S 00000000 2764 2370 1 2380 2353 (NOTLB)
f04d0f4c 00000082 f04d0f34 00000000 cffc18f0 0000260f c782e9a5 00003a7f
f0ba2230 f0ba23bc 03d6d34f 03d6d34f 0004940f fa09bac0 c030b886 f65d4eb8
c035b038 03d6d34f 1d244b3c 00000000 00000005 c031a0b5 c031c25c 000000a8
Call Trace:
[<c030b886>] schedule_timeout+0xf1/0x10c
[<c0129336>] process_timeout+0x0/0x5
[<c012944e>] sys_nanosleep+0x103/0x1a1
[<c030cefb>] syscall_call+0x7/0xb
[<c030007b>] xfrm_state_flush+0x2/0x289
smbd S C0107DF7 3436 2374 2349 (NOTLB)
f04cffb8 00000082 00000001 c0107df7 f0ba33a0 0006ccbc 8803dfb7 00000034
f6a6c600 f6a6c78c 00d1da3c 00cfc97c 00cfc7d0 f04cf000 c012f12b c030cefb
00d1da3c 00000000 08120880 00cfc97c 00cfc7d0 bfed9178 0000001d 0000007b
Call Trace:
[<c0107df7>] handle_IRQ_event+0x25/0x4f
[<c012f12b>] sys_pause+0x14/0x1d
[<c030cefb>] syscall_call+0x7/0xb
dbus-daemon-1 S 00000001 3040 2380 1 2381 2370 (NOTLB)
f65eef1c 00000086 c01459c0 00000001 f0b76bd0 000020ae b10881c6 00000035
f0ba2800 f0ba298c 00000000 7fffffff f65eefa0 f39a5ca0 c030b7e5 00000246
f0b4ca80 f65eefa0 f0b64d18 c0306e62 c03822c0 00000145 f39a5cb8 00000001
Call Trace:
[<c01459c0>] __alloc_pages+0xb4/0x29d
[<c030b7e5>] schedule_timeout+0x50/0x10c
[<c0306e62>] unix_poll+0x24/0xa3
[<c02a8f2c>] sock_poll+0x12/0x14
[<c0176db3>] do_pollfd+0x54/0x77
[<c0176e63>] do_poll+0x8d/0xab
[<c0177020>] sys_poll+0x19f/0x24f
[<c0176461>] __pollwait+0x0/0x94
[<c01244b5>] sys_gettimeofday+0x53/0xac
[<c030cefb>] syscall_call+0x7/0xb
dbus-daemon-1 S 00000000 3096 2381 1 2390 2380 (NOTLB)
f04c8cac 00000086 00000000 00000000 00000000 00005927 9626398f 00000034
f0b771a0 f0b7732c 00000000 7fffffff f04c8d1c f7fe6080 c030b7e5 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
Call Trace:
[<c030b7e5>] schedule_timeout+0x50/0x10c
[<c02ae622>] wait_for_packet+0xca/0x114
[<c011ceaf>] autoremove_wake_function+0x0/0x2d
[<c011ceaf>] autoremove_wake_function+0x0/0x2d
[<c02ae7e2>] skb_recv_datagram+0x176/0x18f
[<c02c5ef5>] netlink_recvmsg+0x54/0x1c1
[<c02a86b6>] sock_recvmsg+0xef/0x10c
[<c01458e9>] buffered_rmqueue+0x1c4/0x1e7
[<c015661b>] anon_vma_prepare+0x1f/0x1bf
[<c0151a41>] do_anonymous_page+0x215/0x27f
[<c011ceaf>] autoremove_wake_function+0x0/0x2d
[<c0151b00>] do_no_page+0x55/0x3bf
[<c02a996c>] sys_recvfrom+0xa5/0xf5
[<c015204a>] handle_mm_fault+0xd5/0x1fd
[<c011924d>] do_page_fault+0x1ac/0x4dc
[<c02aa0b5>] sys_socketcall+0x179/0x1dd
[<c030cefb>] syscall_call+0x7/0xb
hald S C035CA9C 2556 2390 1 2399 2381 (NOTLB)
f65e5f1c 00000086 000000d0 c035ca9c cffc01b0 00007310 e7657e44 00003abc
f0b76bd0 f0b76d5c 03d64a69 03d64a69 f65e5fa0 e71f7d00 c030b886 c035a8a8
c035a8a8 03d64a69 1d244b3c 00000000 00000005 c031a0b5 c031c25c 000000a8
Call Trace:
[<c030b886>] schedule_timeout+0xf1/0x10c
[<c0129336>] process_timeout+0x0/0x5
[<c0176e63>] do_poll+0x8d/0xab
[<c0177020>] sys_poll+0x19f/0x24f
[<c0176461>] __pollwait+0x0/0x94
[<c01244b5>] sys_gettimeofday+0x53/0xac
[<c030cefb>] syscall_call+0x7/0xb
[<c030007b>] xfrm_state_flush+0x2/0x289
mingetty S 00000001 3040 2399 1 2400 2390 (NOTLB)
f65e6e94 00000086 0000000a 00000001 f697c130 0001550a e18fa1a8 00000034
f6a6cbd0 f6a6cd5c f04d2000 7fffffff f04d2c64 fffffffb c030b7e5 00000000
ffffffff 00000000 f65e6ed8 f65e6ef8 00000000 f65e6f3c f7c69714 f65e6f74
Call Trace:
[<c030b7e5>] schedule_timeout+0x50/0x10c
[<c01c2327>] inode_has_perm+0x4c/0x54
[<c021f69b>] read_chan+0x319/0xa11
[<c021ff55>] write_chan+0x1c2/0x1d5
[<c011b856>] default_wake_function+0x0/0xc
[<c011b856>] default_wake_function+0x0/0xc
[<c02195ca>] tty_read+0x140/0x1ea
[<c01620b2>] vfs_read+0xb6/0xe2
[<c01622c5>] sys_read+0x3c/0x62
[<c030cefb>] syscall_call+0x7/0xb
[<c030007b>] xfrm_state_flush+0x2/0x289
mingetty S 00000034 3040 2400 1 2401 2399 (NOTLB)
f65e7e94 00000082 c4b810a2 00000034 f7f111a0 0003bf45 ecb0d76b 00000034
f0b77770 f0b778fc f39b6000 7fffffff f39b6c64 fffffffb c030b7e5 00000000
ffffffff 00000000 f65e7ed8 f65e7ef8 00000000 f65e7f3c f7c73d8c f65e7f74
Call Trace:
[<c030b7e5>] schedule_timeout+0x50/0x10c
[<c01c2327>] inode_has_perm+0x4c/0x54
[<c021f69b>] read_chan+0x319/0xa11
[<c021ff55>] write_chan+0x1c2/0x1d5
[<c011b856>] default_wake_function+0x0/0xc
[<c011b856>] default_wake_function+0x0/0xc
[<c02195ca>] tty_read+0x140/0x1ea
[<c01620b2>] vfs_read+0xb6/0xe2
[<c01622c5>] sys_read+0x3c/0x62
[<c030cefb>] syscall_call+0x7/0xb
mingetty S 00000001 3040 2401 1 2402 2400 (NOTLB)
f08c0e94 00000082 0000000a 00000001 f0b76030 0004849f cb166906 00000034
f0b76600 f0b7678c f08f3000 7fffffff f08f3c64 fffffffb c030b7e5 00000000
ffffffff 00000000 f08c0ed8 f08c0ef8 00000000 f08c0f3c f7c774ec f08c0f74
Call Trace:
[<c030b7e5>] schedule_timeout+0x50/0x10c
[<c01c2327>] inode_has_perm+0x4c/0x54
[<c021f69b>] read_chan+0x319/0xa11
[<c021ff55>] write_chan+0x1c2/0x1d5
[<c011b856>] default_wake_function+0x0/0xc
[<c011b856>] default_wake_function+0x0/0xc
[<c02195ca>] tty_read+0x140/0x1ea
[<c01620b2>] vfs_read+0xb6/0xe2
[<c01622c5>] sys_read+0x3c/0x62
[<c030cefb>] syscall_call+0x7/0xb
mingetty S 00000001 3040 2402 1 2403 2401 (NOTLB)
f08c3e94 00000086 0000000a 00000001 f69b18f0 00052d6d d66f7d73 00000034
f0b76030 f0b761bc f1550000 7fffffff f1550c64 fffffffb c030b7e5 00000000
ffffffff 00000000 f08c3ed8 f08c3ef8 00000000 f08c3f3c f7c80b64 f08c3f74
Call Trace:
[<c030b7e5>] schedule_timeout+0x50/0x10c
[<c01c2327>] inode_has_perm+0x4c/0x54
[<c021f69b>] read_chan+0x319/0xa11
[<c021ff55>] write_chan+0x1c2/0x1d5
[<c011b856>] default_wake_function+0x0/0xc
[<c011b856>] default_wake_function+0x0/0xc
[<c02195ca>] tty_read+0x140/0x1ea
[<c01620b2>] vfs_read+0xb6/0xe2
[<c01622c5>] sys_read+0x3c/0x62
[<c030cefb>] syscall_call+0x7/0xb
mingetty S 00000001 3040 2403 1 6851 2402 (NOTLB)
f08c6e94 00000082 0000000a 00000001 f6a6cbd0 0003ccd9 e18e4c9e 00000034
f08e97f0 f08e997c f1866000 7fffffff f1866c64 fffffffb c030b7e5 00000000
ffffffff 00000000 f08c6ed8 f08c6ef8 00000000 f08c6f3c f7c832c4 f08c6f74
Call Trace:
[<c030b7e5>] schedule_timeout+0x50/0x10c
[<c01c2327>] inode_has_perm+0x4c/0x54
[<c021f69b>] read_chan+0x319/0xa11
[<c021ff55>] write_chan+0x1c2/0x1d5
[<c011b856>] default_wake_function+0x0/0xc
[<c011b856>] default_wake_function+0x0/0xc
[<c02195ca>] tty_read+0x140/0x1ea
[<c01620b2>] vfs_read+0xb6/0xe2
[<c01622c5>] sys_read+0x3c/0x62
[<c030cefb>] syscall_call+0x7/0xb
[<c030007b>] xfrm_state_flush+0x2/0x289
mingetty S 00000001 3040 6851 1 5922 2403 (NOTLB)
f6999e94 00000082 0000000a 00000001 f7f11770 00027873 8e10f662 00000049
f69b18f0 f69b1a7c f695b000 7fffffff f695bc64 fffffffb c030b7e5 00000000
ffffffff 00000000 f6999ed8 f6999ef8 00000000 f6999f3c f69c6d8c f6999f74
Call Trace:
[<c030b7e5>] schedule_timeout+0x50/0x10c
[<c01c2327>] inode_has_perm+0x4c/0x54
[<c021f69b>] read_chan+0x319/0xa11
[<c021ff55>] write_chan+0x1c2/0x1d5
[<c011b856>] default_wake_function+0x0/0xc
[<c011b856>] default_wake_function+0x0/0xc
[<c02195ca>] tty_read+0x140/0x1ea
[<c01620b2>] vfs_read+0xb6/0xe2
[<c01622c5>] sys_read+0x3c/0x62
[<c030cefb>] syscall_call+0x7/0xb
pickup S 00000246 2824 5271 2316 7087 2325 (NOTLB)
d354ceb0 00000086 c127cfc0 00000246 cfeb97f0 0000428f 32f517f9 00003ab7
f08e80b0 f08e823c 03d76956 03d76956 00000000 d354cf74 c030b886 c037d940
c035b048 03d76956 1d244b3c 00000000 00000005 c031a0b5 c031c25c 000000a8
Call Trace:
[<c030b886>] schedule_timeout+0xf1/0x10c
[<c0129336>] process_timeout+0x0/0x5
[<c01768e1>] do_select+0x347/0x378
[<c0176461>] __pollwait+0x0/0x94
[<c0176c05>] sys_select+0x2e0/0x43a
[<c01243db>] sys_time+0xf/0x58
[<c030cefb>] syscall_call+0x7/0xb
mgetty S C01459C0 2764 5922 1 6438 6851 (NOTLB)
e6214eb0 00000082 c035ca9c c01459c0 00000001 0001cb89 5cfc24c7 000038ed
f697c130 f697c2bc 03eec231 03eec231 00000000 e6214f74 c030b886 c035b350
c035b350 03eec231 1d244b3c 00000000 00000005 c031a0b5 c031c25c 000000a8
Call Trace:
[<c01459c0>] __alloc_pages+0xb4/0x29d
[<c030b886>] schedule_timeout+0xf1/0x10c
[<c0129336>] process_timeout+0x0/0x5
[<c01768e1>] do_select+0x347/0x378
[<c0176461>] __pollwait+0x0/0x94
[<c0176c05>] sys_select+0x2e0/0x43a
[<c030cefb>] syscall_call+0x7/0xb
crond S F786F540 3024 5980 2340 5981 (NOTLB)
d157aee8 00000086 d157af74 f786f540 c01c2327 00010b87 7ce582b9 00003951
f78a77f0 f78a797c f3e0e0ac d157af10 d157af04 f3e0e02c c016fc5c 00000000
f78a77f0 c011ceaf d157af1c d157af1c 0001b1a6 b7fff000 f3dc2e58 00000000
Call Trace:
[<c01c2327>] inode_has_perm+0x4c/0x54
[<c016fc5c>] pipe_wait+0x70/0xa9
[<c011ceaf>] autoremove_wake_function+0x0/0x2d
[<c011ceaf>] autoremove_wake_function+0x0/0x2d
[<c016febc>] pipe_readv+0x227/0x2a2
[<c016ff53>] pipe_read+0x1c/0x20
[<c01620b2>] vfs_read+0xb6/0xe2
[<c01622c5>] sys_read+0x3c/0x62
[<c030cefb>] syscall_call+0x7/0xb
run-parts S 00000004 2600 5981 5980 6437 (NOTLB)
d11b2f28 00000082 f08e1be0 00000004 cfd1d8f0 000076ab 28d43399 00003954
f69b0780 f69b090c f69b0824 fffffe00 f69b0780 00000000 c0123c14 00030002
d11b2fc4 bfe476d4 d0ff0030 d11b2000 00000001 00000004 ffffffff 00000000
Call Trace:
[<c0123c14>] do_wait+0x25a/0x435
[<c011b856>] default_wake_function+0x0/0xc
[<c012ef43>] sys_rt_sigaction+0xdd/0xf2
[<c011b856>] default_wake_function+0x0/0xc
[<c0123e82>] sys_wait4+0x27/0x2a
[<c0123e98>] sys_waitpid+0x13/0x17
[<c030cefb>] syscall_call+0x7/0xb
awk S F08E1BE0 3192 6437 5981 (NOTLB)
d0809ee8 00000086 d0809f74 f08e1be0 cff6c130 000082f3 2e8bea3b 000039ae
d0ff0030 d0ff01bc f3e0e414 d0809f10 d0809f04 f3e0e394 c016fc5c 00000000
d0ff0030 c011ceaf d0809f1c d0809f1c b7fff000 d0809fc4 d0acb918 00000000
Call Trace:
[<c016fc5c>] pipe_wait+0x70/0xa9
[<c011ceaf>] autoremove_wake_function+0x0/0x2d
[<c011ceaf>] autoremove_wake_function+0x0/0x2d
[<c016febc>] pipe_readv+0x227/0x2a2
[<c016ff53>] pipe_read+0x1c/0x20
[<c01620b2>] vfs_read+0xb6/0xe2
[<c01622c5>] sys_read+0x3c/0x62
[<c030cefb>] syscall_call+0x7/0xb
atom S 00000004 2576 6438 1 6850 6440 5922 (NOTLB)
d167bf28 00000086 f3e27a00 00000004 cfeb97f0 000117de 5c2feef4 00003957
d0ff0600 d0ff078c d0ff06a4 fffffe00 d0ff0600 00000000 c0123c14 00030002
d167bfc4 bfe4a6f4 cfe54600 d167b000 00000001 00000004 ffffffff 00000000
Call Trace:
[<c0123c14>] do_wait+0x25a/0x435
[<c011b856>] default_wake_function+0x0/0xc
[<c012ef43>] sys_rt_sigaction+0xdd/0xf2
[<c011b856>] default_wake_function+0x0/0xc
[<c0123e82>] sys_wait4+0x27/0x2a
[<c0123e98>] sys_waitpid+0x13/0x17
[<c030cefb>] syscall_call+0x7/0xb
backup_local- S 00000004 2764 6440 1 6808 6441 6438 (NOTLB)
d085cf28 00000086 f10c8e40 00000004 cfeb97f0 0000fad2 4914056f 00003954
d08657f0 d086597c d0865894 fffffe00 d08657f0 00000000 c0123c14 00030002
d085cfc4 bff2e5d4 cfdc0dd0 d085c000 00000001 00000004 ffffffff 00000000
Call Trace:
[<c0123c14>] do_wait+0x25a/0x435
[<c011b856>] default_wake_function+0x0/0xc
[<c012ef43>] sys_rt_sigaction+0xdd/0xf2
[<c011b856>] default_wake_function+0x0/0xc
[<c0123e82>] sys_wait4+0x27/0x2a
[<c0123e98>] sys_waitpid+0x13/0x17
[<c030cefb>] syscall_call+0x7/0xb
backup_remote S 00000004 2764 6441 1 6727 6443 6440 (NOTLB)
d67e7f28 00000086 f1867d40 00000004 cfeb9220 0000f6a0 3e9c21e1 00003954
d0b26dd0 d0b26f5c d0b26e74 fffffe00 d0b26dd0 00000000 c0123c14 00030002
d67e7fc4 bfe5a654 cfe55770 d67e7000 00000001 00000004 ffffffff 00000000
Call Trace:
[<c0123c14>] do_wait+0x25a/0x435
[<c011b856>] default_wake_function+0x0/0xc
[<c012ef43>] sys_rt_sigaction+0xdd/0xf2
[<c011b856>] default_wake_function+0x0/0xc
[<c0123e82>] sys_wait4+0x27/0x2a
[<c0123e98>] sys_waitpid+0x13/0x17
[<c030cefb>] syscall_call+0x7/0xb
backup_remote S 00000004 2600 6443 1 6623 6449 6441 (NOTLB)
cfc1af28 00000086 f6a07520 00000004 cfeb9220 0000f0be 354885d8 00003954
d0864c50 d0864ddc d0864cf4 fffffe00 d0864c50 00000000 c0123c14 00030002
cfc1afc4 bff04774 cf9191a0 cfc1a000 00000001 00000004 ffffffff 00000000
Call Trace:
[<c0123c14>] do_wait+0x25a/0x435
[<c011b856>] default_wake_function+0x0/0xc
[<c012ef43>] sys_rt_sigaction+0xdd/0xf2
[<c011b856>] default_wake_function+0x0/0xc
[<c0123e82>] sys_wait4+0x27/0x2a
[<c0123e98>] sys_waitpid+0x13/0x17
[<c030cefb>] syscall_call+0x7/0xb
hirdet S 00000004 2576 6449 1 6821 6451 6443 (NOTLB)
d03f0f28 00000082 f0bbd180 00000004 cfeb97f0 000117c7 4b2384c2 00003954
d02d0700 d02d088c d02d07a4 fffffe00 d02d0700 00000000 c0123c14 00030002
d03f0fc4 bff984a4 cfeb97f0 d03f0000 00000001 00000004 ffffffff 00000000
Call Trace:
[<c0123c14>] do_wait+0x25a/0x435
[<c011b856>] default_wake_function+0x0/0xc
[<c012ef43>] sys_rt_sigaction+0xdd/0xf2
[<c011b856>] default_wake_function+0x0/0xc
[<c0123e82>] sys_wait4+0x27/0x2a
[<c0123e98>] sys_waitpid+0x13/0x17
[<c030cefb>] syscall_call+0x7/0xb
mailmkk S 00000004 2764 6451 1 6849 6452 6449 (NOTLB)
cfc58f28 00000086 f0820e60 00000004 d0b27970 00012bfd d06721be 00003956
cfd1d8f0 cfd1da7c cfd1d994 fffffe00 cfd1d8f0 00000000 c0123c14 00030002
cfc58fc4 bfe3dfd4 cf919770 cfc58000 00000001 00000004 ffffffff 00000000
Call Trace:
[<c0123c14>] do_wait+0x25a/0x435
[<c011b856>] default_wake_function+0x0/0xc
[<c012ef43>] sys_rt_sigaction+0xdd/0xf2
[<c011b856>] default_wake_function+0x0/0xc
[<c0123e82>] sys_wait4+0x27/0x2a
[<c0123e98>] sys_waitpid+0x13/0x17
[<c030cefb>] syscall_call+0x7/0xb
novell S 00003954 2576 6452 1 6830 6453 6451 (NOTLB)
cfd11f28 00000082 5138e6ed 00003954 f6a6d1a0 00010790 5138e6ed 00003954
cfd1d320 cfd1d4ac cfd1d3c4 fffffe00 cfd1d320 00000000 c0123c14 00030002
cfd11fc4 bff9f7a4 cffc18f0 cfd11000 00000001 00000004 ffffffff 00000000
Call Trace:
[<c0123c14>] do_wait+0x25a/0x435
[<c011b856>] default_wake_function+0x0/0xc
[<c012ef43>] sys_rt_sigaction+0xdd/0xf2
[<c011b856>] default_wake_function+0x0/0xc
[<c0123e82>] sys_wait4+0x27/0x2a
[<c0123e98>] sys_waitpid+0x13/0x17
[<c030cefb>] syscall_call+0x7/0xb
pallasz S 00000004 2576 6453 1 6831 6454 6452 (NOTLB)
cfd32f28 00000082 f18674c0 00000004 cff6c130 00010786 595d5245 00003954
cfd1cd50 cfd1cedc cfd1cdf4 fffffe00 cfd1cd50 00000000 c0123c14 00030002
cfd32fc4 bfe45824 cffc0d50 cfd32000 00000001 00000004 ffffffff 00000000
Call Trace:
[<c0123c14>] do_wait+0x25a/0x435
[<c011b856>] default_wake_function+0x0/0xc
[<c012ef43>] sys_rt_sigaction+0xdd/0xf2
[<c011b856>] default_wake_function+0x0/0xc
[<c0123e82>] sys_wait4+0x27/0x2a
[<c0123e98>] sys_waitpid+0x13/0x17
[<c030cefb>] syscall_call+0x7/0xb
tempo2 S 00000004 2576 6454 1 6829 6455 6453 (NOTLB)
cfd4df28 00000086 f39a53c0 00000004 cff6c130 0001ce28 50de794b 00003954
cfd1c780 cfd1c90c cfd1c824 fffffe00 cfd1c780 00000000 c0123c14 00030002
00000000 c0265fe7 0423e7b8 cfd4d000 00000001 00000004 ffffffff 00000000
Call Trace:
[<c0123c14>] do_wait+0x25a/0x435
[<c0265fe7>] start_request+0x1a0/0x1c1
[<c011b856>] default_wake_function+0x0/0xc
[<c012ef43>] sys_rt_sigaction+0xdd/0xf2
[<c011b856>] default_wake_function+0x0/0xc
[<c0123e82>] sys_wait4+0x27/0x2a
[<c0123e98>] sys_waitpid+0x13/0x17
[<c030cefb>] syscall_call+0x7/0xb
[<c030007b>] xfrm_state_flush+0x2/0x289
ubul S 00000004 2576 6455 1 7002 6456 6454 (NOTLB)
cfd69f28 00000086 f39a54a0 00000004 cf9191a0 000187b2 a8415fc5 000039e6
cfd1c1b0 cfd1c33c cfd1c254 fffffe00 cfd1c1b0 00000000 c0123c14 00030002
cfd69fc4 bfe56314 cffc01b0 cfd69000 00000001 00000004 ffffffff 00000000
Call Trace:
[<c0123c14>] do_wait+0x25a/0x435
[<c011b856>] default_wake_function+0x0/0xc
[<c012ef43>] sys_rt_sigaction+0xdd/0xf2
[<c011b856>] default_wake_function+0x0/0xc
[<c0123e82>] sys_wait4+0x27/0x2a
[<c0123e98>] sys_waitpid+0x13/0x17
[<c030cefb>] syscall_call+0x7/0xb
zeusz S 00003954 2576 6456 1 6818 6705 6455 (NOTLB)
cfd85f28 00000086 4aaa25cb 00003954 cfdc13a0 0000fca6 4aaa25cb 00003954
cfdc1970 cfdc1afc cfdc1a14 fffffe00 cfdc1970 00000000 c0123c14 00030002
cfd85fc4 bfe2fbd4 cffc1320 cfd85000 00000001 00000004 ffffffff 00000000
Call Trace:
[<c0123c14>] do_wait+0x25a/0x435
[<c011b856>] default_wake_function+0x0/0xc
[<c012ef43>] sys_rt_sigaction+0xdd/0xf2
[<c011b856>] default_wake_function+0x0/0xc
[<c0123e82>] sys_wait4+0x27/0x2a
[<c0123e98>] sys_waitpid+0x13/0x17
[<c030cefb>] syscall_call+0x7/0xb
rdiff-backup R running 1920 6623 6443 6838 (NOTLB)
ncpmount S 00003AB6 3112 6705 1 6770 6456 (NOTLB)
cff28eb0 00000086 7798132f 00003ab6 cf854dd0 000009e6 7798132f 00003ab6
cff6d870 cff6d9fc cfded480 7fffffff 00000000 cff28f74 c030b7e5 cf9f5780
00000000 cff28f58 cf9f5780 cfa54c98 c017648e 00000246 00000246 d05aa800
schedule_timeout+0x50/0x10c schedule_timeout+0xf1/0x10c
[<c0129336>] process_timeout+0x0/0x5
[<c02ab707>] sk_wait_data+0x87/0xbb
[<c011ceaf>] autoremove_wake_function+0x0/0x2d
[<c011ceaf>] autoremove_wake_function+0x0/0x2d
[<c02d465a>] tcp_recvmsg+0x379/0x651
[<c02abe91>] sock_common_recvmsg+0x30/0x46
[<c02a86b6>] sock_recvmsg+0xef/0x10c
[<c030b7e5>] schedule_timeout+0x50/0x10c
[<c011adc1>] recalc_task_prio+0x128/0x133
[<c030ad9f>] schedule+0x43f/0x552
[<c011ceaf>] autoremove_wake_function+0x0/0x2d
[<c02a86fd>] kernel_recvmsg+0x2a/0x31
[<f8b9cde5>] cifs_demultiplex_thread+0x11e/0x99c [cifs]
[<f8b9ccc7>] cifs_demultiplex_thread+0x0/0x99c [cifs]
[<c01041d9>] kernel_thread_helper+0x5/0xb
ncpmount S 000000D0 3112 6810 1 6826 6812 (NOTLB)
d3275eb0 00000082 00000246 000000d0 cffc0d50 0000097d 77981cac 00003ab6
cf854dd0 cf854f5c cfee8e80 7fffffff 00000000 d3275f74 c030b7e5 cfb1fa80
00000000 d3275f58 cfb1fa80 cf8cdd18 c017648e 00000246 00000246 d05aa080
Call Trace:
[<c030b7e5>] schedule_timeout+0x50/0x10c
[<c017648e>] __pollwait+0x2d/0x94
[<c01768e1>] do_select+0x347/0x378
[<c0176461>] __pollwait+0x0/0x94
[<c0176c05>] sys_select+0x2e0/0x43a
[<c0175723>] sys_fcntl64+0x6c/0x7d
[<c017572d>] sys_fcntl64+0x76/0x7d
[<c030cefb>] syscall_call+0x7/0xb
[<c030007b>] xfrm_state_flush+0x2/0x289
cifsd S 00003AAD 2340 6811 1 6812 6799 (L-TLB)
d11d6d9c 00000046 8947e102 00003aad cf919770 000005f0 2e399238 00003abc
cfdc13a0 cfdc152c 03d651c8 03d651c8 00000000 d11d6e14 c030b886 c035ade8
f04d3eb8 03d651c8 1d244b3c 00000000 00000005 c031a0b5 c031c25c 000000a8
Call Trace:
[<c030b886>] schedule_timeout+0xf1/0x10c
[<c0129336>] process_timeout+0x0/0x5
[<c02ab707>] sk_wait_data+0x87/0xbb
[<c011ceaf>] autoremove_wake_function+0x0/0x2d
[<c02ae889>] skb_copy_datagram_iovec+0x53/0x1e5
[<c011ceaf>] autoremove_wake_function+0x0/0x2d
[<c02d465a>] tcp_recvmsg+0x379/0x651
[<c02abe91>] sock_common_recvmsg+0x30/0x46
[<c02a86b6>] sock_recvmsg+0xef/0x10c
[<c0107df7>] handle_IRQ_event+0x25/0x4f
[<f885b35e>] scsi_delete_timer+0xc/0x48 [scsi_mod]
[<c011adc1>] recalc_task_prio+0x128/0x133
[<c030ad9f>] schedule+0x43f/0x552
[<c011ceaf>] autoremove_wake_function+0x0/0x2d
[<c02a86fd>] kernel_recvmsg+0x2a/0x31
[<f8b9cde5>] cifs_demultiplex_thread+0x11e/0x99c [cifs]
[<f8b9ccc7>] cifs_demultiplex_thread+0x0/0x99c [cifs]
[<c01041d9>] kernel_thread_helper+0x5/0xb
cifsd S 00003ABC 2604 6812 1 6810 6811 (L-TLB)
d1233d9c 00000046 843db3c4 00003abc cf8553a0 00000d31 5baec11c 00003abd
cff6d2a0 cff6d42c 03d66592 03d66592 00000000 d1233e14 c030b886 c035ae88
f6952eb8 03d66592 1d244b3c 00000000 00000005 c031a0b5 c031c25c 000000a8
Call Trace:
schedule_timeout+0xf1/0x10c
[<c0129336>] process_timeout+0x0/0x5
[<c02ab707>] sk_wait_data+0x87/0xbb
[<c011ceaf>] autoremove_wake_function+0x0/0x2d
[<c02ae889>] skb_copy_datagram_iovec+0x53/0x1e5
[<c011ceaf>] autoremove_wake_function+0x0/0x2d
[<c02d465a>] tcp_recvmsg+0x379/0x651
[<c02abe91>] sock_common_recvmsg+0x30/0x46
[<c02a86b6>] sock_recvmsg+0xef/0x10c
[<c011b72b>] scheduler_tick+0x37f/0x4aa
[<c0129210>] do_timer+0x49/0xb5
[<c010cde3>] timer_interrupt+0x9c/0x181
[<c0107df7>] handle_IRQ_event+0x25/0x4f
[<c0144396>] mempool_alloc+0x7b/0x1f9
[<c011ceaf>] autoremove_wake_function+0x0/0x2d
[<c011ceaf>] autoremove_wake_function+0x0/0x2d
[<c02a86fd>] kernel_recvmsg+0x2a/0x31
[<f8b9cde5>] cifs_demultiplex_thread+0x11e/0x99c [cifs]
[<f8b9ccc7>] cifs_demultiplex_thread+0x0/0x99c [cifs]
[<c01041d9>] kernel_thread_helper+0x5/0xb
rdiff-backup D CFF83D20 1700 6818 6456 (NOTLB)
cff83d58 00000082 00000010 cff83d20 cf8553a0 000fca76 5bf0288f 00003abd
cffc1320 cffc14ac 03d684d6 cff83000 cff83dac c18f1580 f8baa34b cf404380
00000000 00000000 f1380f80 0000d1b2 0000000f 00000000 cffc1320 c011ceaf
Call Trace:
autoremove_wake_function+0x0/0x2d
[<c011ceaf>] autoremove_wake_function+0x0/0x2d
[<f88ca027>] char2uni+0xe/0x1f [nls_utf8]
__cond_resched+0x14/0x3b
[<f8ba6d9b>] cifs_revalidate+0x1a6/0x2d6 [cifs]
[<f8ba2553>] cifs_d_revalidate+0xb/0x35 [cifs]
[<c017136b>] do_lookup+0x70/0x8f
[<c0171db7>] link_path_walk+0xa2d/0xd8c
[<c0172391>] path_lookup+0xff/0x12f
[<c01724d5>] __user_walk+0x21/0x51
[<c016ca23>] vfs_lstat+0x11/0x37
[<c016d018>] sys_lstat64+0xf/0x23
[<c01281b0>] ptrace_setsiginfo+0x1be/0x1c0
[<c010884e>] do_IRQ+0x239/0x242
[<c030cefb>] syscall_call+0x7/0xb
[<c030007b>] xfrm_state_flush+0x2/0x289
rdiff-backup R running 1476 6821 6449 (NOTLB)
ncpmount S 000000D0 3112 6826 1 6810 (NOTLB)
d0eb9eb0 00000082 00000246 000000d0 cfdc0230 000008ad 85dcc468 00003a71
cfeb8c50 cfeb8ddc cf516980 7fffffff 00000000 d0eb9f74 c030b7e5 cf516d80
00000000 d0eb9f58 cf516d80 cfa1d798 c017648e 00000246 00000246 cff66800
schedule_timeout+0x50/0x10c
[<c017648e>] __pollwait+0x2d/0x94
[<c01768e1>] do_select+0x347/0x378
[<c0176461>] __pollwait+0x0/0x94
[<c0176c05>] sys_select+0x2e0/0x43a
[<c0175723>] sys_fcntl64+0x6c/0x7d
[<c017572d>] sys_fcntl64+0x76/0x7d
[<c030cefb>] syscall_call+0x7/0xb
rdiff-backup R running 1172 6829 6454 (NOTLB)
rdiff-backup D 00003ABD 1912 6830 6452 (NOTLB)
cffd1eec 00000086 588b9723 00003abd f697c700 0000019a 588b9ad8 00003abd
cffc18f0 cffc1a7c f7fd6e00 cffd1f18 f7fd6ea4 002130fc f889e7a8 00000000
00000000 cffc18f0 c011ceaf cffd1f24 cffd1f24 00000000 f7fd6efc 00000003
Call Trace:
autoremove_wake_function+0x0/0x2d
[<c011ceaf>] autoremove_wake_function+0x0/0x2d
[<f88985e9>] journal_stop+0x37b/0x42f [jbd]
[<f88986b9>] journal_force_commit+0x1c/0x21 [jbd] direct_strncpy_from_user+0x3e/0x5d
[<c01412b1>] filemap_fdatawrite+0xf/0x13
[<c0163aec>] sys_fsync+0x95/0xd6
[<c030cefb>] syscall_call+0x7/0xb
[<c030007b>] xfrm_state_flush+0x2/0x289
rdiff-backup R running 1628 6831 6453 (NOTLB)
ssh S 00003ABD 2428 6836 6727 (NOTLB)
dfa9aeb0 00000082 49cc20c5 00003abd cfe55770 000003ca 49cc2d68 00003abd
cf854230 cf8543bc cf2d7680 7fffffff 00000000 dfa9af74 c030b7e5 cf9ea698
c017648e 00000246 ce6cb580 dfa9af58 cf9ea698 c02d1fed 00000000 00000082
schedule_timeout+0x50/0x10c
[<c017648e>] __pollwait+0x2d/0x94
[<c02d1fed>] tcp_poll+0x31/0x144
[<c01768e1>] do_select+0x347/0x378
[<c0176461>] __pollwait+0x0/0x94
[<c0176c05>] sys_select+0x2e0/0x43a
[<c030cefb>] syscall_call+0x7/0xb
[<c030007b>] xfrm_state_flush+0x2/0x289
ssh S C035CA9C 2544 6838 6623 (NOTLB)
e7f4eeb0 00000082 000000d0 c035ca9c cf919770 0000123e 5a529d9a 00003a82
cfdc0230 cfdc03bc cda89880 7fffffff 00000000 e7f4ef74 c030b7e5 cf9eac98
c017648e 00000246 cdf0fa80 e7f4ef58 cf9eac98 c02d1fed 000007b0 00000082
Call Trace:
[<c030b7e5>] schedule_timeout+0x50/0x10c
[<c017648e>] __pollwait+0x2d/0x94
[<c02d1fed>] tcp_poll+0x31/0x144
[<c01768e1>] do_select+0x347/0x378
[<c0176461>] __pollwait+0x0/0x94
[<c0176c05>] sys_select+0x2e0/0x43a
[<c030cefb>] syscall_call+0x7/0xb
[<c030007b>] xfrm_state_flush+0x2/0x289
rdiff-backup S F0820E60 1836 6849 6451 6867 (NOTLB)
cf8eaee8 00000086 cf8eaf74 f0820e60 cffc18f0 0003b947 f0afe825 00003abc
cf919770 cf9198fc cff9e5c8 cf8eaf10 cf8eaf04 cff9e548 c016fc5c 00000000
cf919770 c011ceaf cf8eaf1c cf8eaf1c 00000000 f6e06034 00000001 00000000
Call Trace:
[<c016fc5c>] pipe_wait+0x70/0xa9
[<c011ceaf>] autoremove_wake_function+0x0/0x2d
[<c011ceaf>] autoremove_wake_function+0x0/0x2d
[<c016febc>] pipe_readv+0x227/0x2a2
[<c016ff53>] pipe_read+0x1c/0x20
[<c01620b2>] vfs_read+0xb6/0xe2
[<c01622c5>] sys_read+0x3c/0x62
[<c030cefb>] syscall_call+0x7/0xb
rdiff-backup R running 2120 6850 6438 6869 (NOTLB)
ssh S C035CA9C 2340 6867 6849 (NOTLB)
e6f9eeb0 00000082 000000d0 c035ca9c cffc18f0 000047cb f0f80be1 00003abc
cf918bd0 cf918d5c ccc19c80 7fffffff 00000000 e6f9ef74 c030b7e5 cf9ea898
c017648e 00000246 ce6cb080 e6f9ef58 cf9ea898 c02d1fed 000003d8 00000082
Call Trace:
[<c030b7e5>] schedule_timeout+0x50/0x10c
[<c017648e>] __pollwait+0x2d/0x94
[<c02d1fed>] tcp_poll+0x31/0x144
[<c01768e1>] do_select+0x347/0x378
[<c0176461>] __pollwait+0x0/0x94
[<c0176c05>] sys_select+0x2e0/0x43a
[<c030cefb>] syscall_call+0x7/0xb
[<c030007b>] xfrm_state_flush+0x2/0x289
ssh S 0000395D 2648 6869 6850 (NOTLB)
c32e5eb0 00000086 2095eedb 0000395d cfe54600 000051e3 20960484 0000395d
cfeb80b0 cfeb823c c59eda80 7fffffff 00000000 c32e5f74 c030b7e5 cda78318
c017648e 00000246 f0b43080 c32e5f58 cda78318 c02d1fed 00000000 00000082
Call Trace:
[<c030b7e5>] schedule_timeout+0x50/0x10c
[<c017648e>] __pollwait+0x2d/0x94
[<c02d1fed>] tcp_poll+0x31/0x144
[<c01768e1>] do_select+0x347/0x378
[<c0176461>] __pollwait+0x0/0x94
[<c0176c05>] sys_select+0x2e0/0x43a
[<c030cefb>] syscall_call+0x7/0xb
rdiff-backup S F39A54A0 2008 7002 6455 7003 (NOTLB)
f57c8ee8 00000082 f57c8f74 f39a54a0 cf8553a0 000002dd 5a3b5875 00003abd
cffc01b0 cffc033c f7cab77c f57c8f10 f57c8f04 f7cab6fc c016fc5c 00000000
cffc01b0 c011ceaf f57c8f1c f57c8f1c 00000000 00000000 f5393a18 00000000
Call Trace:
[<c016fc5c>] pipe_wait+0x70/0xa9
[<c011ceaf>] autoremove_wake_function+0x0/0x2d
[<c011ceaf>] autoremove_wake_function+0x0/0x2d
[<c016febc>] pipe_readv+0x227/0x2a2
[<c016ff53>] pipe_read+0x1c/0x20
[<c01620b2>] vfs_read+0xb6/0xe2
[<c01622c5>] sys_read+0x3c/0x62
[<c030cefb>] syscall_call+0x7/0xb
[<c030007b>] xfrm_state_flush+0x2/0x289
ssh S 00003ABD 2604 7003 7002 (NOTLB)
c63f8eb0 00000082 5a3b2f32 00003abd cffc01b0 00000500 5a3b3bd2 00003abd
cfe54030 cfe541bc cfaf5d80 7fffffff 00000000 c63f8f74 c030b7e5 e9770598
c017648e 00000246 cf886a80 c63f8f58 e9770598 c02d1fed 000003d8 00000082
Call Trace:
[<c030b7e5>] schedule_timeout+0x50/0x10c
[<c017648e>] __pollwait+0x2d/0x94
[<c02d1fed>] tcp_poll+0x31/0x144
[<c01768e1>] do_select+0x347/0x378
[<c0176461>] __pollwait+0x0/0x94
[<c0176c05>] sys_select+0x2e0/0x43a
[<c030cefb>] syscall_call+0x7/0xb
proxymap S 00000246 2276 7087 2316 7088 5271 (NOTLB)
e155feb0 00000082 c10841a0 00000246 cf9191a0 00001ef1 3982b7ea 00003aa6
cfa0c680 cfa0c80c ce771480 7fffffff 00000000 e155ff74 c030b7e5 00000001
f0524380 00000000 e155ff58 f0524380 f0528718 c017648e 00000246 cff66300
Call Trace:
[<c030b7e5>] schedule_timeout+0x50/0x10c
[<c017648e>] __pollwait+0x2d/0x94
[<c01768e1>] do_select+0x347/0x378
[<c0176461>] __pollwait+0x0/0x94
[<c0176c05>] sys_select+0x2e0/0x43a
[<c030cefb>] syscall_call+0x7/0xb
[<c030007b>] xfrm_state_flush+0x2/0x289
trivial-rewri S 00000246 2276 7088 2316 7128 7087 (NOTLB)
f0e43eb0 00000082 c105f3e0 00000246 f08e8c50 00000896 3e4abd9a 00003aa6
cfa0cc50 cfa0cddc cddada80 7fffffff 00000000 f0e43f74 c030b7e5 00000001
f6946c80 00000000 f0e43f58 f6946c80 f051c118 c017648e 00000246 f0b4c300
Call Trace:
[<c030b7e5>] schedule_timeout+0x50/0x10c
[<c017648e>] __pollwait+0x2d/0x94
[<c01768e1>] do_select+0x347/0x378
[<c0176461>] __pollwait+0x0/0x94
[<c0176c05>] sys_select+0x2e0/0x43a
[<c030cefb>] syscall_call+0x7/0xb
[<c030007b>] xfrm_state_flush+0x2/0x289
smtpd S 00003AA6 2196 7128 2316 7129 7088 (NOTLB)
d2e70eb0 00000082 3ed0320a 00003aa6 f780a1b0 000136ff 3ed291b3 00003aa6
f08e8680 f08e880c 03d64c63 03d64c63 00000000 d2e70f74 c030b886 cb2eeeb8
f6966f54 03d64c63 1d244b3c 00000000 00000005 c031a0b5 c031c25c 000000a8
Call Trace:
[<c030b886>] schedule_timeout+0xf1/0x10c
[<c0129336>] process_timeout+0x0/0x5
[<c01768e1>] do_select+0x347/0x378
[<c0176461>] __pollwait+0x0/0x94
[<c0176c05>] sys_select+0x2e0/0x43a
[<c01243db>] sys_time+0xf/0x58
[<c030cefb>] syscall_call+0x7/0xb
[<c030007b>] xfrm_state_flush+0x2/0x289
cleanup S 00000246 2276 7129 2316 7130 7128 (NOTLB)
cb2eeeb0 00000082 c1376f40 00000246 cffc01b0 0000558e 3ed5b624 00003aa6
f08e8c50 f08e8ddc 03d64c64 03d64c64 00000000 cb2eef74 c030b886 f7f28f4c
d2e70eb8 03d64c64 1d244b3c 00000000 00000005 c031a0b5 c031c25c 000000a8
Call Trace:
[<c030b886>] schedule_timeout+0xf1/0x10c
[<c0129336>] process_timeout+0x0/0x5
[<c01768e1>] do_select+0x347/0x378
[<c0176461>] __pollwait+0x0/0x94
[<c0176c05>] sys_select+0x2e0/0x43a
[<c01243db>] sys_time+0xf/0x58
[<c030cefb>] syscall_call+0x7/0xb
[<c030007b>] xfrm_state_flush+0x2/0x289
local S 00003AA6 2364 7130 2316 7129 (NOTLB)
eb6a9eb0 00000086 affc3a08 00003aa6 cf854230 0000532f affc3a08 00003aa6
d0b26800 d0b2698c 03d653d1 03d653d1 00000000 eb6a9f74 c030b886 c035adf8
c035adf8 03d653d1 1d244b3c 00000000 00000005 c031a0b5 c031c25c 000000a8
Call Trace:
[<c030b886>] schedule_timeout+0xf1/0x10c
[<c0129336>] process_timeout+0x0/0x5
[<c01768e1>] do_select+0x347/0x378
[<c0176461>] __pollwait+0x0/0x94
[<c0176c05>] sys_select+0x2e0/0x43a
[<c01243db>] sys_time+0xf/0x58
[<c030cefb>] syscall_call+0x7/0xb
Mem-info:
DMA per-cpu:
cpu 0 hot: low 2, high 6, batch 1
cpu 0 cold: low 0, high 2, batch 1
Normal per-cpu:
cpu 0 hot: low 32, high 96, batch 16
cpu 0 cold: low 0, high 32, batch 16
HighMem per-cpu:
cpu 0 hot: low 16, high 48, batch 8
cpu 0 cold: low 0, high 16, batch 8
Free pages: 331372kB (256kB HighMem)
Active:62682 inactive:102899 dirty:25948 writeback:0 unstable:0 free:82843 slab:5921 mapped:29439 pagetables:474
DMA free:1276kB min:16kB low:32kB high:48kB active:3884kB inactive:7156kB present:16384kB pages_scanned:0 all_unreclaimable? no
protections[]: 0 0 0
Normal free:329840kB min:936kB low:1872kB high:2808kB active:175308kB inactive:349000kB present:901120kB pages_scanned:0 all_unreclaimable? no
protections[]: 0 0 0
HighMem free:256kB min:128kB low:256kB high:384kB active:71536kB inactive:55440kB present:131072kB pages_scanned:0 all_unreclaimable? no
protections[]: 0 0 0
DMA: 213*4kB 47*8kB 1*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1276kB
Normal: 0*4kB 19006*8kB 8122*16kB 1377*32kB 47*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 329840kB
HighMem: 0*4kB 0*8kB 0*16kB 4*32kB 0*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 256kB
Swap cache: add 40, delete 40, find 0/0, race 0+0
Free swap: 2108440kB
262144 pages of RAM
32768 pages of HIGHMEM
3192 reserved pages
56867 pages shared
0 pages swap cached
