Unable to handle kernel NULL pointer

[Adam Dixon <adamtdixon@xxxxxxxxx>]

Hi,
Running MySQL Cluster datanode (a process of large memory which
checkpoints data to disk periodically) on a 2xXeon Dell PowerEdge
SC1425
(2.6.9-5.ELsmp #1 SMP x86_64), and recieved this notice in
/var/log/messages. The datanode process crashed, and using the
partition after this seemed to hang processes, eg ls /data/ would hang.
Reboot fixed this, boot.log states that the partition was clean,
Dec 19 08:33:51 data3 fsck: /data: recovering journal 
Dec 19 08:33:54 data3 fsck: /data: clean, 9907/6311872 files, 3946561/12619049 blocks 

I posted this info to the cluster list initially,
they seem to think it might be a memory > disk problem or perhaps a
bug? And said you guys might know more about it.


Dec 18 03:18:09 data3 kernel: Unable to handle kernel NULL pointer dereference at 0000000000000020 RIP: 

Dec 18 03:18:09 data3 kernel: <ffffffffa003cfc8>{:jbd:journal_dirty_metadata+71}

Dec 18 03:18:09 data3 kernel: PML4 f4cb067 PGD 2d383067 PMD 0
Dec 18 03:18:09 data3 kernel: Oops: 0000 [1] SMP
Dec 18 03:18:09 data3 kernel: CPU 2
Dec 18 03:18:09 data3 kernel: Modules linked in: md5 ipv6 i2c_dev i2c_core sunrpc dm_mod button battery ac uhci_hcd ehci_hcd e1000 ext3 jbd ata_piix libata sd_mod scsi_mod
Dec 18 03:18:09 data3 kernel: Pid: 11186, comm: ndbd Not tainted 2.6.9-5.ELsmp
Dec 18 03:18:09 data3 kernel: RIP: 0010:[<ffffffffa003cfc8>] <ffffffffa003cfc8>{:jbd:journal_dirty_metadata+71}
Dec 18 03:18:09 data3 kernel: RSP: 0018:000001007c18fbb8  EFLAGS: 00010246
Dec 18 03:18:09 data3 kernel: RAX: 0000000000000000 RBX: 00000101ad1405c0 RCX: 00000000ffffffff
Dec 18 03:18:09 data3 kernel: RDX: 000000000000000f RSI: 00000100447c2ce8 RDI: 0000010024e32820
Dec 18 03:18:09 data3 kernel: RBP: 00000100447c2ce8 R08: 0000000000000000 R09: 0000000000000000
Dec 18 03:18:09 data3 kernel: R10: 0000000000000001 R11: 0000000000000001 R12: 0000000000000000
Dec 18 03:18:09 data3 kernel: R13: 00000100cfe15c00 R14: 0000010024e32820 R15: 000001011cd75130
Dec 18 03:18:09 data3 kernel: FS:  0000000040076960(005b) GS:ffffffff804bf400(0000) knlGS:0000000000000000
Dec 18 03:18:09 data3 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Dec 18 03:18:09 data3 kernel: CR2: 0000000000000020 CR3: 00000000cff0e000 CR4: 00000000000006e0
Dec 18 03:18:09 data3 kernel: Process ndbd (pid: 11186, threadinfo 000001007c18e000, task 00000101ae70c030)
Dec 18 03:18:09 data3 kernel: Stack: 000001012431cc80 0000000000000000 000001011cd75240 00000100cfe43c00
Dec 18 03:18:09 data3 kernel:        0000000000000001 ffffffffa00528c2 00000100447c2ce8 000001007c18fc38
Dec 18 03:18:09 data3 kernel:        0000010024e32820 000001011cd75240
Dec 18 03:18:09 data3 kernel: Call Trace:<ffffffffa00528c2>{:ext3:ext3_mark_iloc_dirty+740}
Dec 18 03:18:09 data3 kernel:        <ffffffffa0052a14>{:ext3:ext3_mark_inode_dirty+65}
Dec 18 03:18:10 data3 kernel:        <ffffffffa003c4c6>{:jbd:journal_start+223} <ffffffffa0054726>{:ext3:ext3_dirty_inode+99}
Dec 18 03:18:10 data3 kernel:        <ffffffff8018fef0>{__mark_inode_dirty+40} <ffffffff80189748>{inode_update_time+182}
Dec 18 03:18:10 data3 kernel:        <ffffffff80155b7b>{generic_file_aio_write_nolock+620}
Dec 18 03:18:10 data3 kernel:        <ffffffff80155c9c>{generic_file_aio_write+126} <ffffffffa004fed0>{:ext3:ext3_file_write+22}
Dec 18 03:18:10 data3 kernel:        <ffffffff80171f8d>{do_sync_write+173} <ffffffff801493ba>{do_futex+590}
Dec 18 03:18:10 data3 kernel:        <ffffffff80132ff0>{autoremove_wake_function+0} <ffffffff80172088>{vfs_write+207}
Dec 18 03:18:10 data3 kernel:        <ffffffff80172170>{sys_write+69} <ffffffff8010ffd2>{system_call+126}
Dec 18 03:18:10 data3 kernel:       
Dec 18 03:18:10 data3 kernel:
Dec 18 03:18:10 data3 kernel: Code: 49 39 5c 24 20 75 4b 41 83 7c 24 0c 02 75 43 49 3b 5d 50 0f
Dec 18 03:18:10 data3 kernel: RIP <ffffffffa003cfc8>{:jbd:journal_dirty_metadata+71} RSP <000001007c18fbb8>
Dec 18 03:18:10 data3 kernel: CR2: 0000000000000020

Any clarification someone can provide would be great,
Thanks,
Adam


_______________________________________________

Ext3-users@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/ext3-users