Hi, As I explained in my first posting that the 'read-only' issue is not for one server, it is happening for few servers which are generally 'oracle' database oriented. Very recently it happned to an 'oracle' application server. For temporary basis , we are re-mounting the file system and also doing fsck. While searching the redhat knowledge base, found the following url, the problem they were explaining it is similar to our issues, https://bugzilla.redhat.com/show_bug.cgi?id=213921 It is telling that it is the bug of the kernel.. Not sure whether we will proceed for the higher version of kernel or not, please advice. Thanks --- tweeks <tweeks@xxxxxxxxxxxxx> wrote: > The EL4 kernel is wacky when it comes the the I/O scheduler locking up and > and > causing ext3 to remount RO. Various hardware hiccups can cause it to go RO. > > And when it does.. you need to tread lightly or you could lose everything. > > If your ext3 filesystem had problems and remounted read-only, I would > strongly > advise /against/ simply fscking it. Often times when your filesystem has > gone RO, it may have been that way for 30 minutes or more. Just rebooting ro > > fscking is a great way to lose everything (i.e. everything being dumped > into /lost+found/" > > Instead, I would recommend: > 1) rebooting into a rescue CD environment (not allowing the rescue > environment > to mount or fsck your filesystems). > 2) Nuke the ext3 journal: > tune2fs -O ^has_journal /dev/<rootfs> > (possibly doing the same for other problem partitions) > 3) Do a fake fsck to see the extent of damage: > fsck -fn /dev/<rootfs> > (after checking things out.. use "-fy" once you're sure that it's safe) > 4) Rebuild the journal w, "tune2fs -j /dev/<rootfs> > (rerun at least once until "clean" result is repeatable) > 5) Mount and check things out, > "mkdir /mnt/tmp && mount -t ext3 /dev/<rootfs> /mnt/tmp" > 6) Gracefully umount & reboot: > "umount /mnt/tmp && shutdown -rf now && exit" > > Tweeks > > On Tuesday 25 September 2007 11:47, Swapana Ghosh wrote: > > Hi Jordi, > > > > Thanks for your reply. I will test the way you suggested. > > > > Thanks > > -swapna > > > > --- Jordi Prats <jprats@xxxxxxxx> wrote: > > > Hi, > > > It seems like what it happened to me. I did this to solve this issue: > > > > > > Mark the filesystem as it does not have a journal (take it to ext2) > > > > > > tune2fs -O ^has_journal /dev/cciss/c0d0p2 > > > > > > fsck it to delete the journal: > > > > > > e2fsck /dev/cciss/c0d0p2 > > > > > > Create the journal (take it back to ext3) > > > > > > tune2fs -j /dev/cciss/c0d0p2 > > > > > > and finaly, remount it. > > > > > > In my case it was with a local disk, but with your SAN disk should be > > > the same. > > > > > > Jordi > > > > > > Swapana Ghosh wrote: > > > > Hi > > > > > > > > In our office environment few servers mostly database servers and > > > > > > yesterday it > > > > > > > happened > > > > for one application server(first time) the partion is getting "read > > > > only". > > > > > > > > I was checking the archives, found may be similar kind of issues in the > > > > 2007-July archives. > > > > But how it has been solved if someone describes me that will be really > > > > > > helpful. > > > > > > > In our case, just at the problem started found the line in log file as > > > > > > follows: > > > > EXT3-fs error (device dm-12): edxt3_find_entry: reading directory > > > > > > #2015496 > > > > > > > offset 2 > > > > > > > > Then one blank line > > > > Then the line is > > > > > > > > Aborting journal on device dm-12. > > > > ext3_abort called > > > > > > > > Ext3-fs error (device dm-12): ext3_journal_start_sb: Detected > > > > aborted journal > > > > Remounting filesysem read-only > > > > > > > > Then the continuous line as follows: > > > > > > > > > > > > EXT3-fs error (device dm-12) in start_transaction: Journal has > > > > aborted > > > > > > > > > > > > > > > > The above message is continuous until we remount the filesystem and > > > > > > partion > > > > > > > becomes > > > > 'read-write'. > > > > > > > > We could not figure it out what is the root cause of the system. > > > > > > > > We are using individual EMC luns and are configured with LVM volume > > > > groups > > > > > > and > > > > > > > then mounted on logical > > > > volumes. > > > > > > > > Here i am giving the server description: > > > > > > > > ____________________________________________________________ > > > > > > > > [root@server ~]# lsmod |grep -i qla > > > > qla2300 130304 0 > > > > qla2xxx_conf 305924 0 > > > > qla2xxx 307448 21 qla2300 > > > > scsi_mod 117709 5 sg,emcp,qla2xxx,cciss,sd_mod > > > > > > > > ____________________________________________________________ > > > > [root@server ~]# cat /etc/modprobe.conf > > > > alias eth0 tg3 > > > > alias eth1 tg3 > > > > alias eth2 e1000 > > > > alias eth3 e1000 > > > > alias eth4 e1000 > > > > alias eth5 e1000 > > > > alias bond0 bonding > > > > alias scsi_hostadapter cciss > > > > options bond0 max_bonds=2 miimon=100 mode=1 > > > > alias scsi_hostadapter1 qla2xxx > > > > alias scsi_hostadapter2 qla2xxx_conf > > > > #alias scsi_hostadapter3 qla6312 > > > > options qla2xxx ql2xmaxqdepth=16 qlport_down_retry=64 > > > > ql2xloginretrycount=30 ql2xfailover=0 ql2xlbType=0 > > > > install qla2xxx /sbin/modprobe qla2xxx_conf; /sbin/modprobe > > > > --ignore-install qla2xxx > > > > remove qla2xxx /sbin/modprobe -r --first-time --ignore-remove qla2xxx > > > > && { /sbin/modprobe -r --ignore-remove qla2xxx_conf; } > > > > ###BEGINPP > > > > include /etc/modprobe.conf.pp > > > > ###ENDPP > > > > ###BEGINPP > > > > include /etc/modprobe.conf.pp > > > > ###ENDPP > > > > ###BEGINPP > > > > include /etc/modprobe.conf.pp > > > > ###ENDPP > > > > > > > > ________________________________________________ > > > > [root@server ~]# rpm -qa |grep -i EMC > > > > EMCpower.LINUX-4.5.1-022 > > > > > > > > ________________________________________________ > > > > [root@server ~]# rpm -qa|grep -i scli > > > > scli-1.06.16-57 > > > > > > > > ________________________________________________ > > > > [root@server ~]# rpm -qa|grep -i nav > > > > naviagentcli-6.19.1.3.0-1 > > > > > > > > ________________________________________________ > > > > product: QLA2312 Fibre Channel Adapter > > > > > > > > ________________________________________________ > > > > [root@server ~]# rpm -qa|grep -i lvm > > > > lvm2-2.02.06-6.0.RHEL4 > > > > system-config-lvm-1.0.19-1.0 > > > > > > > > ________________________________________________ > > > > > > > > If I missed any info, pl. let me know. > > > > > > > > It would be really appreciated if I get some hints to solve the issues > > > > > > > > Thanks in advance > > > > -swapana > > > > ___________________________________________________________________________ > >_________ > > > > > > Boardwalk for $500? In 2007? Ha! Play Monopoly Here and Now (it's > > > > updated > > > > > > for today's economy) at Yahoo! Games. > === message truncated === ____________________________________________________________________________________ Tonight's top picks. What will you watch tonight? Preview the hottest shows on Yahoo! TV. http://tv.yahoo.com/ _______________________________________________ Ext3-users mailing list Ext3-users@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/ext3-users
