Help on ext3 file system corruption issue

"Somasundaram, Arun (IE10)" <Arun.Somasundaram@xxxxxxxxxxxxx> · Tue, 5 Jun 2007 22:01:00 +0530

Hi All,

  I m a novice developer of Linux applications. Recently
I faced a file system corruption. (I guess)

I have a Kernel 2.4.7-10 with ext3 file system in compact
flash. The system was up for 3 months and was running with average load
conditions.

One fine day, it just started sending kernel messages on the
serial console. The message was like this.

EXT3-fs error
(device ide0(3,2)): ext3_get_inode_loc: unab

le to read inode
block - inode=20089, block=81926

EXT3-fs error
(device ide0(3,2)): ext3_get_inode_loc: unable to read inode block

 - inode=20090,
block=81926

EXT3-fs error
(device ide0(3,2)): ext3_get_inode_loc: unable to read inode block

 - inode=20091,
block=81926

EXT3-fs error
(device ide0(3,2)): ext3_get_inode_loc: unable to read inode block

 - inode=20092,
block=81926

EXT3-fs error (device
ide0(3,2)): ext3_get_inode_loc: unable to read inode block

 - inode=20093,
block=81926

EXT3-fs error
(device ide0(3,2)): ext3_get_inode_loc: unable to read inode block

 - inode=20094,
block=81926

EXT3-fs error
(device ide0(3,2)): ext3_get_inode_loc: unable to read inode block

 - inode=20095,
block=81926

EXT3-fs error
(device ide0(3,2)): ext3_get_inode_loc: unable to read inode block

 - inode=20096,
block=81926

EXT3-fs error
(device ide0(3,2)): ext3_get_inode_loc: unable to read inode block

 - inode=20097,
block=81927

EXT3-fs error
(device ide0(3,2)): ext3_get_inode_loc: unable to read inode block

 - inode=20098,
block=81927

EXT3-fs error
(device ide0(3,2)): ext3_get_inode_loc: unable to read inode block

 - inode=20099,
block=81927

Assertion failure in
do_get_write_access() at transaction.c:606: "!(((jh2bh(jh))

->b_state &
(1UL << BH_Lock)) != 0)"

invalid operand:
0000

CPU:   
0

EIP:   
0010:[<d080e9e5>]

EFLAGS: 00010286

eax:
00000021   ebx: c171ce94   ecx: 00000001   edx:
0015991c

esi:
c171ce00   edi: c788cdc0   ebp: cb5f4690   esp:
cf581bf0

ds: 0018  
es: 0018   ss: 0018

Process syslogd
(pid: 608, stackpage=cf581000)

Stack: d0816990
0000025e 00000000 00000000 c171ce00 cf641820 c171ce94 c171ce00

c788cdc0 cb5f4690 d080edb5 c788cdc0 cb5f4690 00000000 00000000 000001ae

cfc86400 cfa670e0
d081a0cb c788cdc0 cfa67140 cf581c60 c1448088 cfa66800

Call Trace:
[<d0816990>] [<d080edb5>] [<d081a0cb>] [<c018526c>]
[<d081c049>]

   [<d081c33d>] [<d081c1d0>] [<d081c990>]
[<d080f3de>] [<d081ec95>] [<d081ed1a>]

[<d081caac>] [<c0133283>] [<d0813ebb>] [<d080e276>]
[<c01339cd>] [<d081ca48>]

[<d081cfa6>] [<d081ca48>] [<c01265c8>] [<d081aa7b>]
[<c0130ec9>] [<d081aa5c>]

[<d080f777>] [<d081ab0f>]
[<c0130fe1>] [<c0106dd3>]

Code: 0f 0b 5b
5e 8b 54 24 24 f6 42 10 04 bb e2 ff ff ff b8 01 00

 hda: read_intr: status=0x51 { DriveReady SeekComplete Error }

hda: read_intr:
error=0x40 { UncorrectableError }, LBAsect=667854, sector=163854

end_request: I/O
error, dev 03:02 (hda), sector 163854

EXT3-fs error
(device ide0(3,2)): ext3_get_inode_loc: unable to read inode block

 - inode=20100,
block=81927

hda: read_intr:
status=0x51 { DriveReady SeekComplete Error }

hda: read_intr:
error=0x40 { UncorrectableError }, LBAsect=667854, sector=163854

end_request: I/O
error, dev 03:02 (hda), sector 163854

EXT3-fs error
(device ide0(3,2)): ext3_get_inode_loc: unable to read inode block

 - inode=20101,
block=81927

hda: read_intr:
status=0x51 { DriveReady SeekComplete Error }

hda: read_intr:
error=0x40 { UncorrectableError }, LBAsect=667854, sector=163854

end_request: I/O
error, dev 03:02 (hda), sector 163854

EXT3-fs error
(device ide0(3,2)): ext3_get_inode_loc: unable to read inode block

 - inode=20102,
block=81927

hda: read_intr:
status=0x51 { DriveReady SeekComplete Error }

hda: read_intr:
error=0x40 { UncorrectableError }, LBAsect=667854, sector=163854

end_request: I/O
error, dev 03:02 (hda), sector 163854

EXT3-fs error
(device ide0(3,2)): ext3_get_inode_loc: unable to read inode block

 - inode=20103,
block=81927

hda: read_intr:
status=0x51 { DriveReady SeekComplete Error }

hda: read_intr:
error=0x40 { UncorrectableError }, LBAsect=667854, sector=163854

end_request: I/O
error, dev 03:02 (hda), sector 163854

EXT3-fs error
(device ide0(3,2)): ext3_get_inode_loc: unable to read inode block

 - inode=20104,
block=81927

30916

Some more information on this system: Wondering if this
could have some deep impact on this issue.

 LILO boots the 2.4.7-10 kernel
     image with option ide=nodma. Can this have any impact on these errors?
 The system has a postgresql
     database which writes data to the maximum of 1 record per 5 second. That
     much data writes will it do.
 On restart, the fsck in the
     bootup scipts (rc.sysinit)  could not resolve this, 

It said.

Checking filesystems

Could this be a
zero-length partition?

fsck.ext3: Attempt
to read block from filesystem resulted in short read while tr

ying to open
/dev/hda2

/dev/hda3:
recovering journal

/dev/hda3: clean,
81/125488 files, 35684/500472 blocks

Checking all file
systems.

[/sbin/fsck.ext3 --
/tmp] fsck.ext3 -a /dev/hda2

[/sbin/fsck.ext3 --
/tmp2] fsck.ext3 -a /dev/hda3

[FAILED]

*** An error
occurred during the file system check.

*** Dropping you to
a shell; the system will reboot

*** when you leave
the shell.

Give root password
for maintenance

(or type Control-D
for normal startup):

I went ahead and further gave root password and ran the
command. e2fsck -a -c -C 0 /dev/hda2

It said:

e2fsck: Attempt to read block from
filesystem resulted in short read while trying to open /dev/hda2

Could this be a zero-length partition?

Please give your advice, as this problem has become a big-bang
show stopper for our product.

Your advice will be very helpful for me to go ahead with
this issue.

Thanks in advance,

Arun S

_______________________________________________
Ext3-users mailing list
Ext3-users@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/ext3-users