Hmmm - I mount my firewall-scripts volume like this now in fstab: localhost:/firewall-scripts /firewall-scripts glusterfs defaults,_netdev 0 0 But gluster volume info still looks like this on both nodes: [root at chicago-fw1 system]# gluster volume info Volume Name: firewall-scripts Type: Replicate Volume ID: 239b6401-e873-449d-a2d3-1eb2f65a1d4c Status: Started Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: 192.168.253.1:/gluster-fw1 Brick2: 192.168.253.2:/gluster-fw2 [root at chicago-fw1 system]# Seems to me, my Gluster volume would still have some dependency on networking, even though I mount it with localhost in fstab, right? In all our trial and error, the only thing that's worked so far is putting in that 30 second delay in my custom rc.local file. So instead of fighting this, why not try my hand at a systemd service file named, say, glustermount.service. So here is the glustermout.service file I made. It runs a script named /etc/rc.d/glustermount.sh pasted in below. [root at chicago-fw1 rc.d]# cd /usr/lib/systemd/system [root at chicago-fw1 system]# more glustermount.service # This unit is Greg's attempt to mount my Gluster filesystem. [Unit] Description=Set up Gluster mounts ConditionFileIsExecutable=/etc/rc.d/glustermount.sh After=network.target glusterd.service [Service] Type=oneshot ExecStart=/etc/rc.d/glustermount.sh TimeoutSec=0 RemainAfterExit=no SysVStartPriority=99 [Install] WantedBy=multi-user.target [root at chicago-fw1 system]# Here is glustermount.sh as it exists right now. Really just a renamed rc.local I've been using the whole time, now with some debug echo lines. [root at chicago-fw1 rc.d]# more glustermount.sh #!/bin/sh # # This script will be executed *after* all the other init scripts. # You can put your own initialization stuff in here if you don't # want to do the full Sys V style init stuff. # # Note removed by default starting in Fedora 16. touch /var/lock/subsys/local #*********************************** # Local stuff below date echo "Sleeping 30 seconds." ### sleep 30 date echo "Making sure the Gluster stuff is mounted" echo "Mounted before mount -av" df -h mount -av echo "Mounted after mount -av" df -h # The fstab mounts happen early in startup, then Gluster starts up later. # By now, Gluster should be up and running and the mounts should work. # That _netdev option is supposed to account for the delay but doesn't seem # to work right. echo "Starting up firewall common items" ## /firewall-scripts/etc/rc.d/common-rc.local [root at chicago-fw1 rc.d]# I should see mounts before and after when I look at a systemctl status. But I don't. And systemctl status shows my /firewall-scripts filesystem successfully mounted, but when I go to look with df -h, it's not mounted. Yet another midnight mystery - take a look. [root at chicago-fw1 system]# systemctl start glustermount.service [root at chicago-fw1 system]# systemctl status glustermount.service -n 50 glustermount.service - Set up Gluster mounts Loaded: loaded (/usr/lib/systemd/system/glustermount.service; enabled) Active: inactive (dead) since Tue 2013-07-16 00:27:36 CDT; 5s ago Process: 2464 ExecStart=/etc/rc.d/glustermount.sh (code=exited, status=0/SUCCESS) Jul 16 00:27:35 chicago-fw1 glustermount.sh[2464]: Tue Jul 16 00:27:35 CDT 2013 Jul 16 00:27:35 chicago-fw1 glustermount.sh[2464]: Sleeping 30 seconds. Jul 16 00:27:35 chicago-fw1 glustermount.sh[2464]: Tue Jul 16 00:27:35 CDT 2013 Jul 16 00:27:35 chicago-fw1 glustermount.sh[2464]: Making sure the Gluster stuff is mounted Jul 16 00:27:35 chicago-fw1 glustermount.sh[2464]: Mounted before mount -av Jul 16 00:27:35 chicago-fw1 glustermount.sh[2464]: Filesystem Size Used Avail Use% Mounted on Jul 16 00:27:35 chicago-fw1 glustermount.sh[2464]: /dev/mapper/fedora-root 14G 3.9G 8.7G 31% / Jul 16 00:27:35 chicago-fw1 glustermount.sh[2464]: devtmpfs 990M 0 990M 0% /dev Jul 16 00:27:35 chicago-fw1 glustermount.sh[2464]: tmpfs 996M 0 996M 0% /dev/shm Jul 16 00:27:35 chicago-fw1 glustermount.sh[2464]: tmpfs 996M 888K 996M 1% /run Jul 16 00:27:35 chicago-fw1 glustermount.sh[2464]: tmpfs 996M 0 996M 0% /sys/...roup Jul 16 00:27:35 chicago-fw1 glustermount.sh[2464]: tmpfs 996M 0 996M 0% /tmp Jul 16 00:27:35 chicago-fw1 glustermount.sh[2464]: /dev/sda2 477M 87M 365M 20% /boot Jul 16 00:27:35 chicago-fw1 glustermount.sh[2464]: /dev/sda1 200M 9.4M 191M 5% /boot/efi Jul 16 00:27:35 chicago-fw1 glustermount.sh[2464]: /dev/mapper/fedora-gluster--fw1 7.9G 33M 7.8G 1% /gluster-fw1 Jul 16 00:27:35 chicago-fw1 glustermount.sh[2464]: extra arguments at end (ignored) Jul 16 00:27:36 chicago-fw1 glustermount.sh[2464]: / : ignored Jul 16 00:27:36 chicago-fw1 glustermount.sh[2464]: /boot : already mounted Jul 16 00:27:36 chicago-fw1 glustermount.sh[2464]: /boot/efi : already mounted Jul 16 00:27:36 chicago-fw1 glustermount.sh[2464]: /gluster-fw1 : already mounted Jul 16 00:27:36 chicago-fw1 glustermount.sh[2464]: swap : ignored Jul 16 00:27:36 chicago-fw1 glustermount.sh[2464]: /firewall-scripts : successfully mounted Jul 16 00:27:36 chicago-fw1 glustermount.sh[2464]: Mounted after mount -av Jul 16 00:27:36 chicago-fw1 glustermount.sh[2464]: Filesystem Size Used Avail Use% Mounted on Jul 16 00:27:36 chicago-fw1 systemd[1]: Started Set up Gluster mounts. [root at chicago-fw1 system]# df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/fedora-root 14G 3.9G 8.7G 31% / devtmpfs 990M 0 990M 0% /dev tmpfs 996M 0 996M 0% /dev/shm tmpfs 996M 888K 996M 1% /run tmpfs 996M 0 996M 0% /sys/fs/cgroup tmpfs 996M 0 996M 0% /tmp /dev/sda2 477M 87M 365M 20% /boot /dev/sda1 200M 9.4M 191M 5% /boot/efi /dev/mapper/fedora-gluster--fw1 7.9G 33M 7.8G 1% /gluster-fw1 [root at chicago-fw1 system]# Finally, running the same glustermount.sh script by hand works just fine. It **only** breaks when run from systemd. Here is what happens when I run it by hand. Compare below to what happens above when I tell systemd to run it. [root at chicago-fw1 system]# [root at chicago-fw1 system]# /etc/rc.d/glustermount.sh Tue Jul 16 00:23:15 CDT 2013 Sleeping 30 seconds. Tue Jul 16 00:23:15 CDT 2013 Making sure the Gluster stuff is mounted Mounted before mount -av Filesystem Size Used Avail Use% Mounted on /dev/mapper/fedora-root 14G 3.9G 8.7G 31% / devtmpfs 990M 0 990M 0% /dev tmpfs 996M 0 996M 0% /dev/shm tmpfs 996M 888K 996M 1% /run tmpfs 996M 0 996M 0% /sys/fs/cgroup tmpfs 996M 0 996M 0% /tmp /dev/sda2 477M 87M 365M 20% /boot /dev/sda1 200M 9.4M 191M 5% /boot/efi /dev/mapper/fedora-gluster--fw1 7.9G 33M 7.8G 1% /gluster-fw1 / : ignored /boot : already mounted /boot/efi : already mounted /gluster-fw1 : already mounted swap : ignored extra arguments at end (ignored) /firewall-scripts : successfully mounted Mounted after mount -av Filesystem Size Used Avail Use% Mounted on /dev/mapper/fedora-root 14G 3.9G 8.7G 31% / devtmpfs 990M 0 990M 0% /dev tmpfs 996M 0 996M 0% /dev/shm tmpfs 996M 888K 996M 1% /run tmpfs 996M 0 996M 0% /sys/fs/cgroup tmpfs 996M 0 996M 0% /tmp /dev/sda2 477M 87M 365M 20% /boot /dev/sda1 200M 9.4M 191M 5% /boot/efi /dev/mapper/fedora-gluster--fw1 7.9G 33M 7.8G 1% /gluster-fw1 localhost:/firewall-scripts 7.6G 19M 7.2G 1% /firewall-scripts Starting up firewall common items [root at chicago-fw1 system]# df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/fedora-root 14G 3.9G 8.7G 31% / devtmpfs 990M 0 990M 0% /dev tmpfs 996M 0 996M 0% /dev/shm tmpfs 996M 888K 996M 1% /run tmpfs 996M 0 996M 0% /sys/fs/cgroup tmpfs 996M 0 996M 0% /tmp /dev/sda2 477M 87M 365M 20% /boot /dev/sda1 200M 9.4M 191M 5% /boot/efi /dev/mapper/fedora-gluster--fw1 7.9G 33M 7.8G 1% /gluster-fw1 localhost:/firewall-scripts 7.6G 19M 7.2G 1% /firewall-scripts [root at chicago-fw1 system]# - Greg
