It took me a few hours (mainly because it has been some years that I played around with netboot and such), but I got it: It needs some cleanup and still gives some errors during the boot process, but I can boot a clean, fresh debian lenny via PXE directly from a glusterfs Storage as a root-filesystem. Of course, you need a dhcpd and tftpd running somewhere so the netboot-server can get its ip, kernel and initrd. this is a very short explanation, as I said it still needs some clean-up and I will hopefully end up with a simple script or debian-package that doeas all the work for you: you need a debian installation to boot: debootstrap lenny $INSTALLDIR http://ftp.de.debian.org/debian chroot $INSTALLDIR echo "deb http://www.backports.org/debian lenny-backports contrib main non-free" >>/etc/apt/sources.list apt-get update apt-get install glusterfs-client debian-backports-keyring vim rsync linux-image-amd64 dnsutils Add a hook to have initramfs copy all necessary stuff to the initrd: /etc/initramfs-tools/hooks/glusterfs ---cut--- #!/bin/sh PREREQ="" prereqs() { echo "$PREREQ" } case $1 in prereqs) prereqs exit 0 ;; esac . /usr/share/initramfs-tools/hook-functions # Begin real processing below this line copy_exec /sbin/mount.glusterfs copy_exec /usr/sbin/glusterfs copy_exec /sbin/ifconfig copy_exec /usr/bin/host test -d ${DESTDIR}/usr/lib/glusterfs/ || mkdir -p ${DESTDIR}/usr/lib/glusterfs/ cp -ar /usr/lib/gluster* ${DESTDIR}/usr/lib/ cp /lib/libnss_* /lib/libresolv* /lib/ld-linux-* /lib/libc.* ${DESTDIR}/lib ---cut--- Now, add a short script to the initrd that will be called automatically when the boot=glusterfs kernel parameter is set (via pxe) /etc/initramfs-tools/scripts/glusterfs ---cut--- mountroot() { echo 192.168.200.206 testserv1 >>/etc/hosts echo 192.168.200.207 linux-netboot >>/etc/hosts echo 192.168.200.206 >>/etc/resolv.conf hostname linux-netboot configure_networking mkdir -p /var/log/glusterfs sh /sbin/mount.glusterfs ${NFSROOT} ${rootmnt} } ---cut--- run (in the chroot): update-initramfs -u to create a new initrd. PXE-config on the server: prompt 1 default lenny timeout 10 label lenny kernel vmlinuz append vga=794 initrd=initrd boot=glusterfs nfsroot=192.168.200.206 ip=dhcp (the kernel and initrd are symlinks to the glusterfs share that holds the root-filesystem:) lrwxrwxrwx 1 root root 24 18. Jun 20:29 /var/lib/tftpboot/initrd -> /data/netboot/initrd.img lrwxrwxrwx 1 root root 21 18. Jun 20:29 /var/lib/tftpboot/vmlinuz -> /data/netboot/vmlinuz Setting it up this way makes it even possible to do even kernel-updates via apt-get on the netboot-client without having to touch anything on the dhcp/pxe-server. more to come... I need to finish some other important work first, but I will hopefully be able to clean everything up by the end of next week and have a howto and some script or package ready for others that are interested. Jan Am 18.06.2010 09:57, schrieb Jan: > Hello Benjamin, > > I think there should be more elegant ways to build the initrd. > > I set up diskless servers that got their root-directory via nfs a few > years ago. > They shared the same installation (rootdir) and we had scripts in the > initrd to mount special directories under /var based on the hostname. > It was a little messy doing kernel-updates and updating everything. > > Debian builds its initrd automatically after the installation of a new > kernel, based on your hardware and configuration. The tool used for > that (initramfs) has a lot of hooks to add your own modules or scripts. > I am sure it is possible to build a simple debian-package that adds > those scripts to your installation. Afterwards, you should even be > able to make a kernel-upgrade on the server although it is a net install. > Setting up a new installation should be as simple as starting > debootstrap (sets up a blank debian-installation (any architecture) in > a subfolder, can also install custom packages listed on the command > line), and then configure the tftp-server. > On the diskless server, you should have all possibilities (including > installation of packages, kernel-updates) that you have on any other > server that has its own disk. > > I hope I'll have the time to have a closer look at this next week. > > Jan > > > Am 17.06.2010 17:16, schrieb Benjamin Hudgens: >> Here were the notes for us in Debian. Since these notes are mildly out >> of context hopefully they simply give you a starting point. Rsync is >> the tool we use to slurp the debian image. You manage gluster as you >> would on a normal install. However, you manage the configs and such on >> a copy at (X) location which is then slurped into ram on a boot. For >> our dumb storage boxes we dynamically detect drives that need >> filesystems, format, then add to gluster configs. Thus our server >> configs get generated on-the-fly. This allows us to maintain a single >> master instance of the OS image and deploy storage boxes quickly. >> >> ############################## >> >> Summary: >> >> We want to run a slim install of Debian in ram. Controlling how we >> mount Gluster is simple. You manage this as you would a regular >> install. Reboot any machine and the changes take effect. You can blow >> away a machine at any time since the OS on our storage machines are >> irrelevant and pull from our master copy. >> >> Below remember that ${rootmnt} is pooled from kernel args for the PXE >> boot. >> >> Unpackage the initrd: >> gzip -dc< ../initrd.img-2.6.30-2-686.netboot | cpio -i >> >> Edit ./init >> Modifications start at line 205: >> >> ------------ >> ## Make a ramdisk >> mkdir /ram >> mount -t tmpfs -o size=2G tmpfs /ram >> >> ##Move the libraries over since rsync was failing initially. This just >> grabs all the libraries from root >> mount ${rootmnt}/lib /lib >> mount ${rootmnt}/usr/lib /usr/lib >> >> ##Copy the filesystem into ram >> ${rootmnt}/usr/bin/rsync -rav ${rootmnt}/ /ram/ >> >> ##Change the rootmnt point to our new location - the rest of the script >> will take care of booting us up >> rootmnt=/ram >> ------ >> >> Now re-create the initrd image: >> >> find . -print0 | cpio -0 -H newc -ov | gzip -c> >> ../initrd.img-2.6.30-2-686.netboot >> >> ################################ >> >> >> >> >> -----Original Message----- >> From: gluster-users-bounces at gluster.org >> [mailto:gluster-users-bounces at gluster.org] On Behalf Of Jan >> Sent: Thursday, June 17, 2010 6:30 AM >> To: gluster-users at gluster.org >> Subject: Re: Netboot / PXE-Boot from glusterfs? >> >> Am 17.06.2010 13:15, schrieb Benjamin Hudgens: >>> Hello Jan, >>> >>> Our company took the approach of slurping our OS into a ram drive and >>> then mounting file system points from Gluster. The OS becomes >>> expendable. In our case (large amounts of dumb storage machines) this >>> is okay. We were itching to get away from NFS. Boot time is slow >> while >>> it reads directly from network -> ram. However, the final result is >> an >>> OS that is extremely fast and no NFS dependency. >>> >>> Obviously this approach is only applicable in certain situations. >>> >>> >> Hello Benjamin, >> >> Thanks, that's an interesting idea - as long as the root filesystem is >> not too big. >> But even then, one might be able to split it up in the most important >> files and directories that are needed to boot and mount the >> glusterfs-directories. >> This needs some time to figure out what are the minimum files needed to >> boot until the glusterfs-directories are available - and a minor change >> in the initrd (mount NFS, create ramdisk, copy files). But definitely >> worth a try. >> >> thanks >> Jan >> _______________________________________________ >> Gluster-users mailing list >> Gluster-users at gluster.org >> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users >> _______________________________________________ >> Gluster-users mailing list >> Gluster-users at gluster.org >> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
