-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hi All,
~ I have a problem related to GFS deployment on a SAN volume,
~ SCENARIO:
~ Clustered Mail Server application
~ Environment: ~ RedHat ES 3.0 ~ Redhat Cluster Suite ~ Cluster file system - GFS (ver 6.0) on ex3fs
~ DELL PowerEdge 1550 / 2650 / 1650 / 1750 ~ EMC CX series ~ HBA Qlogix 23xx
~ SET-UP ~ DNS--> load balancer(s) --> cluster node + file lock management --> SAN ~ (mutli LUN) ~ single file lock manager in the cluster (one among the cluster nodes) ~ 3 nodes in backend cluster to handle POP3 and SMTP requests ~ data stored in SAN, block size 128K, read / write cache enabled
~ all the nodes have direct access to SAN through fibre channel
~ In the migration process , when there were 3 nodes , all the systems ~ were running smoothly ~ but ! ~ when I added some more load (mailboxes) with 2 more backend nodes, after ~ some time 2-3 nodes (any of the nodes in cluster) hanged with the ~ following error:
~ Oct 24 11:04:29 c5 kernel: GFS: Assertion failed on line 599 of file ~ inode.c ~ Oct 24 11:04:29 c5 kernel: GFS: assertion: "(tmp_gh->gh_flags ~ &GL_LOCAL_EXCL) || !(gh->gh_flags & GL_LOCAL_EXCL)" ~ Oct 24 11:04:29 c5 kernel: GFS: time = 1095145469 Oct 24 11:04:29 c5 ~ kernel: GFS: fsid=310farm:gfs01.2: glock = (4, 0) ~ Oct 24 11:04:29 c5 kernel: ~ Oct 24 11:04:29 c5 kernel: Kernel panic: GFS: Record message above and ~ reboot.
~ (LOCK MANAGER STILL KEEPS RUNNING and other nodes function properly) ~ I tried with different configurations of Lock manager but failed to get ~ it right.
~ Is the problem with
~ GFS (any limitations ?????????) ~ I/O ~ block size ~ lock management ~ or (my architecture itself)
~ How can i resolve this issue? ~ I am unable to find out proper reason of this. ~ Can someone of you who's had hands on this help me out?
~ Thanks for all those wonderful (non-witty) answers ! ~ Thorough solutions on the above win a linux friend :-) !
- - -- Regards Ritesh Agrawal -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.2 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iD8DBQFBglSEFoz+P95jnTIRAuFsAJ9zK0zu8O6F5tGqZ6I1NSTGnLInzgCbBVU9 5qzDkbXxS/DSRGF6ABGYm0Y= =gFBU -----END PGP SIGNATURE-----
