IBM BladeCenter HS20 blades

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]



We have purchased a product with our SAN to take a snapshot of one LUN 
so we can image that to another LUN for super fast installs. Therefore, 
I wanted to avoid having to set up xCat.

The problem with booting the CentOS 4 DVD was that the CDROM was not 
being detected. Since the CDROM in the media tray is USB, I switched 
over to another blade, and back, and the install then found the CDROM, 
and the rest went perfectly.

Thanks for your help on this.

Brendon Colby

John Burk wrote:
> This problem also exhibits in RHEL3-U4, so CentOS 3.4 inherited it.
> 
> CentOS 3.4 is treated as rhas3 by xcat.  The first time you run 'nodeset 
> <host> install' for a particular OS, the /tftpboot/xcat/rhas3/<arch> 
> tree gets populated with files, including install.gz, which is a 
> compressed disk image which in turn contains all the kernel modules, 
> including tg3.
> 
> 1.) First thing you need to do is delete (or rename) the 
> /tftpboot/xcat/rhas3/<arch>/ directory.
> 
> 2.) Next is to fix a minor bug in xcat's 'mkinstall' located in 
> $XCATROOT/install/rhas3/ARCH/bin/.  I posted the fix to the xcat user's 
> list, and have included the hacked mkinstall as an attachment.  Please 
> note that this mkinstall is for 'x86', but it should be pretty 
> straightforward to see how to make the change for x86_64 if that's your 
> arch.
> 
> <<  cut&paste'd from my post in the xcat user's list  >>
> I went the second route and dug around inside 
> $XCATROOT/install/rhas3/ARCH/bin/mkinstall.  Turns out that there is a 
> fix already inside this script and a module already built, but that it 
> only will be applied to RHEL3-U4 and not CentOS 3.4
> 
> It's because of the file test at line 375 of mkinstall: the version 
> string in the kernel rpm that ships on the RHEL3-U4 cd's is 2.4.21-27, 
> but the CentOS-3.4 kernel version string is 2.4.21-27.0.1.  This version 
> mismatch is echo'd at lines 383 and 386 also.
> << eo snip >>
> 
> 3.) 'cd' to the $XCATROOT/install/rhas3/ARCH/ directory.  The fix only 
> gets applied if you're in this directory, as the path the to bcm5700.o 
> is relative - Egan Ford probably knows why he did it this way, I'm just 
> hacking on his script and wanted to muck with it as little as possible.
> 
> 4.) run 'nodeset <some_CentOS3.4_host> install'.  Because there is not 
> an install.gz in the /tftpboot/xcat/rhas3/<arch> directory, it will be 
> built, and because you're in the $XCATROOT/install/rhas3/ARCH/ 
> directory, the pre-built bcm5700.o module which lives in 
> $XCATROOT/install/rhas3/ARCH/data/27.EL will get put into the install.gz 
> as tg3.o.
> 
> 5.) 'rreset' your blades and watch them properly install CentOS 3.4 via 
> nfs...
> 
> John Burk
> 
> Sr. Technical Director
> Mainframe Entertainment
> 
> 604.628.1019
> 
> 
> 
> 
> Niswonger Charles M Contr AFRL/MNGG wrote:
> 
>> John,
>>
>>         I'm running 14 older HS20's right now and have 14 of the new 
>> 8843 3.6 GHz EM64T blades coming in.  Right now we're running SuSE 9.2 
>> and xcat, but we've seen the same tg3 bug that you mentioned.  I'm 
>> curious as to the workaround that you used for CentOS.  With the 
>> multi-level load needed to install the bootable image onto the blade 
>> we still don't have a satisfactory workaround...
>>
>> Mike Niswonger, Systems Engineer
>> charles.niswonger@xxxxxxxxxxxx
>> Air Force Research Lab / MNGG (KHILS),
>> SYColeman Corp.
>> 101 W Eglin Blvd, Bldg. 13, Rm. 115
>> Eglin AFB, FL 32542-5427
>> (850)882-4446, x1217
>> (850)882-4451, fax
>>
>> -----Original Message-----
>> From: centos-bounces@xxxxxxxxxx [mailto:centos-bounces@xxxxxxxxxx] On 
>> Behalf Of John Burk
>> Sent: Friday, April 15, 2005 1:42 PM
>> To: CentOS mailing list
>> Subject: Re:  IBM BladeCenter HS20 blades
>>
>> I have 60 HS20's running CentOS 3.4, but I do the install via the xcat 
>> toolset over nfs.
>>
>> Have you looked at xcat yet to manage the os on your blades?  Well 
>> worth the time to setup, written primarily by IBM, and free.
>>
>> http://www.xcat.org/
>>
>> BTW, what processors are in your blades?  I'm running the 3.2GHz 
>> Nocona's, if your are similar you'll probably need to use these kernel 
>> parameters in your grub.conf:
>>
>>     acpi=noirq noexec=off noapic
>>
>> And if you do go with nfs installs via xcat, the tg3 driver in 3.4 is 
>> broken.  E-mail me and I'll send you the work-around.  It's already in 
>> xcat but it takes a bit of finesse to apply the fix.
>>
>> John Burk
>>
>> Sr. Technical Director
>> Mainframe Entertainment
>>
>> 604.628.1019
>>
>>
>>
>>
>> Brendon Colby wrote:
>>
>> > Greetings,
>> >
>> > We have purchased an IBM BladeCenter and I am in the process of
>> > testing Linux installation on these things (boot off SAN i.e. qla2300
>> > driver, not using internal drives). My distro of choice is Debian,
>> > however, since I'm really not interested in trying to hand compile all
>> > the drivers, I decided to try CentOS (which I'm so far very impressed
>> > with).
>> >
>> > On boot, as with the Debian Sarge disk, the CDROM disappears. The
>> > installation media complains that it cannot find the CD. I'm testing
>> > using the DVD ISO of CentOS 4. Has anyone been successful in getting
>> > CentOS 3/4 installed on a BladeCenter server? I noticed back in 11/04
>> > there were a few posts on it.
>> >
>> > Thanks.
>> >
>> > Brendon Colby
>> > _______________________________________________
>> > CentOS mailing list
>> > CentOS@xxxxxxxxxx
>> > http://lists.centos.org/mailman/listinfo/centos
>> >
>> >
>> _______________________________________________
>> CentOS mailing list
>> CentOS@xxxxxxxxxx
>> http://lists.centos.org/mailman/listinfo/centos
>>
> 
> ------------------------------------------------------------------------
> 
> #!/bin/ksh
> #egan@xxxxxxxxxx
> #See $XCATROOT/README for license.
> #
> 
> if [ -z "$XCATROOT" ]
> then
> 	if [ -r /etc/sysconfig/xcat ]
> 	then
> 		. /etc/sysconfig/xcat
> 	else
> 		if [ -r /etc/rc.config ]
> 		then
> 			. /etc/rc.config
> 		fi
> 	fi
> fi
> 
> if [ -z "$XCATROOT" ]
> then
> 	if [ -r "$(dirname $0)/../lib/xcatroot" ]
> 	then
> 		. $(dirname $0)/../lib/xcatroot
> 	fi
> fi
> 
> if [ -z "$XCATROOT" ]
> then
> 	echo "$(basename $0): env XCATROOT not defined!" >&2
> 	exit 1
> fi
> 
> if [ ! -d "$XCATROOT" ]
> then
> 	echo "$(basename $0): XCATROOT $XCATROOT does not exist!" >&2
> 	exit 1
> fi
> 
> . $XCATROOT/lib/functions
> 
> function usage {
> 	echo "\nUsage: ${MYNAME} [singlenode]\n"
> }
> 
> needhelp $* && usage >&2 && exit 0
> 
> if [ "$#" != "1" ]
> then
> 	usage >&2
> 	exit 1
> fi
> 
> NODE=$1
> NODENAME=$1
> 
> if ! isanode $1 2>/dev/null
> then
> 	echo "${MYNAME}: $1 is not a node" >&2
> 	exit 1
> fi
> 
> if NODETYPE=$(tabdb $NODETYPETAB $NODE 3)
> then
> 	:
> else
> 	echo "${MYNAME}: $NODE nodetype image not defined in $NODETYPETAB" >&2
> 	exit 1
> fi
> 
> if OSVER=$(tabdb $NODETYPETAB $NODE 1)
> then
> 	:
> else
> 	echo "${MYNAME}: $NODE nodetype osver not defined in $NODETYPETAB" >&2
> 	exit 1
> fi
> 
> if ARCH=$(tabdb $NODETYPETAB $NODE 2)
> then
> 	:
> else
> 	echo "${MYNAME}: $NODE nodetype arch not defined in $NODETYPETAB" >&2
> 	exit 1
> fi
> 
> if NODERES=$(whatismyres2 $NODE)
> then
> 	RESNAME=$NODERES
> else
> 	echo "${MYNAME}: $NODE noderes not defined in $NODERESTAB" >&2
> 	exit 1
> fi
> 
> MYOSVER=rhas3
> MYARCH=x86
> 
> if [ "$OSVER" != "$MYOSVER" -o "$ARCH" != "$MYARCH" ]
> then
> 	echo "${MYNAME}: Install directory does not match $NODE in $NODETYPETAB"
> 	exit 1
> fi
> 
> if INSTALLDIR=$(tabdb $SITETAB installdir 1)
> then
> 	:
> else
> 	echo "$MYNAME: installdir not defined in $SITETAB"
> 	exit 1
> fi
> 
> DISTDIR=$OSVER/$ARCH
> CFDIR=scripts/$OSVER/$ARCH
> PXET=install.pxe
> GRUBT=install.grub
> ELILOT=elilo.conf
> KERNEL=installz
> INITRD=install.gz
> 
> FILES="$INSTALLDIR/$DISTDIR/images/pxeboot/initrd.img $INSTALLDIR/$DISTDIR/images/pxeboot/vmlinuz /usr/share/terminfo/x/xterm $TFTPDIR/$TFTPXCATROOT $TFTPDIR"
> 
> if [ -r $TFTPDIR/$TFTPXCATROOT/$OSVER/$ARCH/$KERNEL -a -r  $TFTPDIR/$TFTPXCATROOT/$OSVER/$ARCH/$INITRD ]
> then
> 	DOIMAGE=0
> else
> 	DOIMAGE=1
> fi
> 
> EXIT=0
> for i in $FILES
> do
> 	if [ ! -r $i ]
> 	then
> 		echo "${MYNAME}: $i missing"
> 		EXIT=1
> 	fi
> 	if [ "$EXIT" = "1" ]
> 	then
> 		exit 1
> 	fi
> done
> 
> if [ -r $XCATPREFIX/tftp/pxelinux.0 ]
> then
> 	mkdir $TFTPDIR >/dev/null 2>&1
> 	cp -f $XCATPREFIX/tftp/pxelinux.0 $TFTPDIR
> else
> 	echo "$MYNAME: $XCATPREFIX/tftp/pxelinux.0 missing"
> 	exit 1
> fi
> 
> if [ -r $XCATPREFIX/tftp/default ]
> then
> 	mkdir $TFTPDIR/pxelinux.cfg >/dev/null 2>&1
> 	cp -f $XCATPREFIX/tftp/default $TFTPDIR/pxelinux.cfg/default
> else
> 	echo "$MYNAME: $XCATPREFIX/tftp/default missing"
> 	exit 1
> fi
> 
> MASTER=$(tabdb $SITETAB master 1) \
> 	|| echo "${MYNAME}: master not defined in $SITETAB"
> MASTER_IP=$(nametoip $MASTER)
> 
> #MASTER_IPS=$(
> #	IPS=""
> #	for i in $(dnsnr "@$MASTER(-eth[0-9])?")
> #	do
> #		if IP=$(nametoip $i)
> #		then
> #			IPS="$IPS $IP"
> #		fi
> #	done
> #	print $IPS
> #)
> 
> MASTER_IPS=$(
> 	IPS=""
> 	for i in $(ifconfig | egrep '^eth[0-9]' | awk '{print $1}')
> 	do
> 		IP=$(ifconfig $i | perl -pi -e 's/ +/:/g' | grep inet:addr: | awk -F: '{print $4}')
> 		IPS="$IPS $IP"
> 	done
> 	print $IPS
> )
> 
> if CONFIG_SERVER=$(tabdb $NODERESTAB $RESNAME $noderes_config_server)
> then
> 	CONFIG_IP=$(nametoip $CONFIG_SERVER)
> else
> 	CONFIG_IP=$MASTER_IP
> 	CONFIG_SERVER=$MASTER
> fi
> 
> RSHC=$(tabdb $SITETAB rsh 1) \
> 	|| echo "${MYNAME}: rsh not defined in $SITETAB"
> 
> if [ "$(basename $RSHC)" = "rsh" ]
> then
> 	RSH="Y"
> else
> 	RSH="N"
> fi
> 
> SERIAL=$(tabdb $NODERESTAB $RESNAME $noderes_serial) \
> 	|| echo "${MYNAME}: serial defined in $NODERESTAB"
> 
> INSTALL_NFS=$(tabdb $NODERESTAB $RESNAME $noderes_nfs_install) \
> 	|| echo "${MYNAME}: nfs not defined in $NODERESTAB"
> INSTALL_NFS_IP=$(nametoip $INSTALL_NFS)
> 
> if [ "$INSTALL_NFS_IP" = "noip" ]
> then
> 	echo "Invalid Install NFS for $NODENAME"
> fi
> 
> INSTALL_NIC=$(tabdb $NODERESTAB $RESNAME $noderes_netdevice) \
> 	|| echo "${MYNAME}: install_nic not defined in $NODERESTAB"
> 
> INSTALL_DIR=$(tabdb $NODERESTAB $RESNAME $noderes_install_dir) \
> 	|| echo "${MYNAME}: nfs dir not defined in $NODERESTAB"
> 
> INSTALL_SRC_DIR=$INSTALL_DIR/$DISTDIR
> INSTALL_CF_DIR=$INSTALL_DIR/$CFDIR
> 
> mkdir -p $INSTALL_CF_DIR
> 
> FILE=$INSTALLDIR/$CFDIR/$NODETYPE-$NODERES
> INSTALL_CF_FILE=$NODETYPE-$NODERES
> 
> if [ ! -r $NODETYPE.tmpl ]
> then
> 	if [ ! -r base/$NODETYPE.tmpl ]
> 	then
> 		echo "${MYNAME}: cannot read template $NODETYPE.tmpl"
> 		exit 1
> 	else
> 		TMPLDIR="base"
> 	fi
> else
> 	TMPLDIR="."
> fi
> 
> export MASTER
> export MASTER_IP
> export MASTER_IPS
> export CONFIG_IP CONFIG_SERVER
> export RSH
> export OSVER
> export ARCH
> export NODE
> export NODERES
> export NODENAME
> export NODETYPE
> export INSTALL_NFS
> export INSTALL_NFS_IP
> export INSTALL_SRC_DIR
> export INSTALL_CF_DIR
> export INSTALL_CF_FILE
> export KERNEL
> export INITRD
> 
> perl -pi -e 's/\\\n$//' <$TMPLDIR/$NODETYPE.tmpl | LANG=C subvars.pl >$FILE
> 
> mkdir -p $TFTPDIR/$TFTPXCATROOT/$OSVER/$ARCH
> 
> if [ -r "$PXET" ]
> then
> 	PXEF=$TFTPDIR/$TFTPXCATROOT/$OSVER/$ARCH/$NODETYPE-$NODERES.pxe
> 	perl -pi -e 's/\\\n$//' <$PXET | LANG=C subvars.pl >$PXEF
> 	if [ "$SERIAL" = "0" -o "$SERIAL" = "1" ]
> 	then
> 		:
> 	else
> 		perl -pi -e "s/^SERIAL/#SERIAL/g" $PXEF
> #		perl -pi -e "s/(console[^ ]* )//" $PXEF
> 		perl -pi -e "s/(console[^ ]* ){2}//" $PXEF
> 	fi
> fi
> 
> if [ -r "$GRUBT" ]
> then
> 	GRUBF=$TFTPDIR/$TFTPXCATROOT/$OSVER/$ARCH/$NODETYPE-$NODERES.grub
> 	perl -pi -e 's/\\\n$//' <$GRUBT | LANG=C subvars.pl >$GRUBF
> 	if [ "$SERIAL" = "0" -o "$SERIAL" = "1" ]
> 	then
> 		:
> 	else
> 		perl -pi -e "s/^serial/#serial/g" $GRUBF
> 		perl -pi -e "s/^terminal/#terminal/g" $GRUBF
> #		perl -pi -e "s/(console[^ ]* )//" $GRUBF
> 		perl -pi -e "s/(console[^ ]* ){2}//" $GRUBF
> 	fi
> fi
> 
> if [ -r "$ELILOT" ]
> then
> 	ELILOC=$TFTPDIR/$TFTPXCATROOT/$OSVER/$ARCH/$NODETYPE-$NODERES.elilo
> 	perl -pi -e 's/\\\n$//' <$ELILOT | LANG=C subvars.pl >$ELILOC
> 	if [ "$SERIAL" = "0" -o "$SERIAL" = "1" ]
> 	then
> 		:
> 	else
> 		perl -pi -e "s/^SERIAL/#SERIAL/g" $ELILOC
> #		perl -pi -e "s/console[^ ]* //" $ELILOC
> 		perl -pi -e "s/(console[^ ]* ){2}//" $ELILOC
> 	fi
> fi
> 
> if [ "$DOIMAGE" = "0" ]
> then
> 	exit 0
> fi
> 
> cp -f $INSTALLDIR/$DISTDIR/images/pxeboot/vmlinuz $TFTPDIR/$TFTPXCATROOT/$OSVER/$ARCH/$KERNEL
> cp -f $INSTALLDIR/$DISTDIR/images/pxeboot/initrd.img $TFTPDIR/$TFTPXCATROOT/$OSVER/$ARCH/$INITRD
> 
> TMPDIR=/tmp/.mkinstall-$OSVER-$ARCH
> 
> if getlock mkinstall-$OSVER-$ARCH 60
> then
> 	:
> else
> 	echo "${MYNAME} Failed to get $TMPDIR lock"
> 	exit 1
> fi
> 
> rm -rf $TMPDIR
> mkdir -p $TMPDIR
> 
> gunzip -c $TFTPDIR/$TFTPXCATROOT/$OSVER/$ARCH/$INITRD >$TMPDIR/initrd 2>/dev/null
> mkdir -p $TMPDIR/root
> mount -o loop $TMPDIR/initrd $TMPDIR/root
> #mkdir -p $TMPDIR/root/etc/terminfo/x
> #mkdir -p $TMPDIR/root/etc/terminfo/v
> #mkdir -p $TMPDIR/root/etc/terminfo/b
> #cp -f /usr/share/terminfo/x/xterm $TMPDIR/root/etc/terminfo/v/vt100
> #cp -f /usr/share/terminfo/x/xterm $TMPDIR/root/etc/terminfo/b/bterm
> #cp -f /usr/share/terminfo/x/xterm $TMPDIR/root/etc/terminfo/x/xterm
> 
> ###
> #rm -f $TMPDIR/root/etc/terminfo/v/vt100-nav
> 
> perl -pi -e 's/^.*qla2.00.*\n$//' $TMPDIR/root/modules/pcitable
> 
> #MYDIR=$(pwd)
> #cd $TMPDIR
> 
> #gunzip -c root/modules/modules.cgz | cpio -id 2>/dev/null
> 
> #if [ -r $MYDIR/data/e100.o ]
> #then
> #	rm -f 2.4.21-3.ELBOOT/eepro100.o
> #	cp -f $MYDIR/data/e100.o 2.4.21-3.ELBOOT/eepro100.o
> #	cat $MYDIR/data/e100.pcitable >> root/modules/pcitable
> #fi
> 
> #if [ -r $MYDIR/data/bcm5700.o ]
> #then
> #	cp -f $MYDIR/data/bcm5700.o 2.4.21-3.ELBOOT/bcm5700.o
> #	cat $MYDIR/data/bcm5700.pcitable >> root/modules/pcitable
> #	cat $MYDIR/data/bcm5700.module-info >> root/modules/module-info
> #fi
> 
> #if [ -r $MYDIR/data/e1000.o ]
> #then
> #	cp -f $MYDIR/data/e1000.o 2.4.21-3.ELBOOT/e1000.o
> #	cat $MYDIR/data/e1000.pcitable >> root/modules/pcitable
> #	cat $MYDIR/data/e1000.module-info >> root/modules/module-info
> #fi
> 
> #find 2.4.21-3.ELBOOT -type f -print | cpio -o -H crc 2>/dev/null | gzip -c -9 >root/modules/modules.cgz
> 
> #cd $MYDIR
> 
> 
> # default kernel version that ships with RHEL3-U4
> KERNELVER=2.4.21-27
> #
> # try and detect if we're actually installing CentOS instead of RHEL
> # ( tests cut&paste'd from XCATROOT/sbin/copycds )
> #
> if [ -f $INSTALLDIR/$DISTDIR/.discinfo ]
> then
> 	if egrep "1104951135.38" $INSTALLDIR/$DISTDIR/.discinfo >/dev/null 2>&1
> 	then
> 		if grep "i386" $INSTALLDIR/$DISTDIR/.discinfo >/dev/null 2>&1
> 		then
> 			# kernel version in the CentOS-3.4 distro
> 			KERNELVER=2.4.21-27.0.1
> 		fi
> 
> 	fi
> fi
> 
> #
> # fix the broken tg3 driver in RHEL3-U4/CentOS-3.4
> #
> if [ -f $INSTALLDIR/$DISTDIR/RedHat/RPMS/kernel-${KERNELVER}.EL.i686.rpm ]
> then
> 	MYDIR=$(pwd)
> 	cd $TMPDIR
> 	gunzip -c root/modules/modules.cgz | cpio -id 2>/dev/null
> 
> 	if [ -r $MYDIR/data/27.EL/bcm5700.o ]
> 	then
> 		cp -f $MYDIR/data/27.EL/bcm5700.o ${KERNELVER}.ELBOOT/i386/tg3.o
> 	fi
> 
> 	find ${KERNELVER}.ELBOOT -type f -print | cpio -o -H crc 2>/dev/null | gzip -c -9 >root/modules/modules.cgz
> 
> 	cd $MYDIR
> fi
> 
> umount $TMPDIR/root
> gzip -9 -f $TMPDIR/initrd
> cp -f $TMPDIR/initrd.gz $TFTPDIR/$TFTPXCATROOT/$OSVER/$ARCH/$INITRD
> 
> rm -rf $TMPDIR
> 
> freelock mkinstall-$OSVER-$ARCH 0
> 
> cd $TFTPDIR/$TFTPXCATROOT/$OSVER/$ARCH
> 
> chown -f root.root *.gz *.pxe *.nbi *.eb
> chmod -f 444 *.gz *.pxe *.nbi *.eb
> 
> exit 0
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> CentOS mailing list
> CentOS@xxxxxxxxxx
> http://lists.centos.org/mailman/listinfo/centos

[Index of Archives]     [CentOS]     [CentOS Announce]     [CentOS Development]     [CentOS ARM Devel]     [CentOS Docs]     [CentOS Virtualization]     [Carrier Grade Linux]     [Linux Media]     [Asterisk]     [DCCP]     [Netdev]     [Xorg]     [Linux USB]
  Powered by Linux