On Wed, 2009-06-24 at 12:02 -0700, Sunil Mushran wrote:
> Do you have a separate network path for drbd traffic? If you do
> not, then you are probably overloading the network. In this case,
> I believe drbd is unable to replicate the ios fast enough and thus
> is blocking the o2cb disk heartbeat. One workaround is to increase
> the O2CB_HEARTBEAT_THRESHOLD to more than the default of 60 secs.
> Refer to the ocfs2 faq or ocfs2 1.4 user's guide for more on this.
>
I've already modified the O2CB_HEARTBEAT_TRESHOLD to different values
(120, 240 etc), with no changes..
> And if you want to capture the logs, setup netconsole.
>
/dev/console is a serial device connected to a terminal server, so far
the best I got was a partial timestamp before I saw the output of the
reboot again ..
It tries to log .. but doesn't finish writing it :( But mostly there is
no activity at all on the serial console :(
Any other ideas ?
greetings
Kris
> Kris Buytaert wrote:
> > We're trying to setup a dual-primary DRBD environment, with a shared
> > disk with either OCFS2 or GFS. The environment is a Centos 5.3 with
> > DRBD82 (but also tried with DRBD83 from testing) .
> >
> > Setting up a single primary disk and running bonnie++ on it works.
> > Setting up a dual-primary disk, only mounting it on one node (ext3) and
> > running bonnie++ works
> >
> > When setting up ocfs2 on the /dev/drbd0 disk and mounting it on both
> > nodes, basic functionality seems in place but usually less than 5-10
> > minutes after I start bonnie++ as a test on one of the nodes , both
> > nodes power cycle with no errors in the logfiles, just a crash.
> >
> > When at the console at the time of crash it looks like a disk IO (you
> > can type , but actions happen) block happens then a reboot, no panics,
> > no oops , nothing. ( sysctl panic values set to timeouts etc )
> > Setting up a dual-primary disk , with ocfs2 only mounting it on one node
> > and starting bonnie++ causes only that node to crash.
> >
> > On DRBD level I get the following error when that node dissapears
> >
> > drbd0: PingAck did not arrive in time.
> > drbd0: peer( Primary -> Unknown ) conn( Connected -> NetworkFailure )
> > pdsk(UpToDate -> DUnknown )
> > drbd0: asender terminated
> > drbd0: Terminating asender thread
> >
> > That however is an expected error because of the reboot.
> >
> > At first I assumed OCFS2 to be the root of this problem ..so I moved
> > forward and setup an ISCSI target on a 3rd node, and used that device
> > with the same OCFS2 setup. There no crashes occured and bonnie++
> > flawlessly completed it test run.
> >
> > So my attention went back to the combination of DRBD and OCFS
> >
> > I tried both DRBD 8.2 drbd82-8.2.6-1.el5.centos kmod-drbd82-8.2.6-2 and
> > the 83 variant from Centos Testing
> >
> > At first I was trying with the ocfs2 1.4.1-1.el5.i386.rpm verson but
> > upgrading to 1.4.2-1.el5.i386.rpm didn't change the behaviour
> >
> >
> > Anyone has an idea on this ?
> > How can we get more debug info from OCFS2 , apart from heartbeat
> > tracing which doesn't learn me nothing yet .. in order to potentially
> > file a valuable bug report.
> >
>
_______________________________________________
CentOS mailing list
CentOS@xxxxxxxxxx
http://lists.centos.org/mailman/listinfo/centos
