I'm trying to add GlusterFS support to the Open Shared Root project, and
I'm hitting a problem that is a bit difficult to debug. GlusterFS works
fine in the normal full-fat environment (RHEL5), but I can't seem to get
it to mount the volume in the init-root environment.
Things I have checked so far:
All the libraries and dependencies that glusterfs package lists are
included in the initroot:
# rpm -q --requires glusterfs
/bin/sh
/bin/sh
config(glusterfs) = 1.3.12-1
libc.so.6()(64bit)
libc.so.6(GLIBC_2.2.5)(64bit)
libc.so.6(GLIBC_2.3)(64bit)
libc.so.6(GLIBC_2.3.2)(64bit)
libc.so.6(GLIBC_2.3.3)(64bit)
libdl.so.2()(64bit)
libdl.so.2(GLIBC_2.2.5)(64bit)
libfuse.so.2()(64bit)
libfuse.so.2(FUSE_2.4)(64bit)
libfuse.so.2(FUSE_2.5)(64bit)
libfuse.so.2(FUSE_2.6)(64bit)
libglusterfs.so.0()(64bit)
libibverbs.so.1()(64bit)
libibverbs.so.1(IBVERBS_1.0)(64bit)
libibverbs.so.1(IBVERBS_1.1)(64bit)
libpthread.so.0()(64bit)
libpthread.so.0(GLIBC_2.2.5)(64bit)
libpthread.so.0(GLIBC_2.3.2)(64bit)
rpmlib(CompressedFileNames) <= 3.0.4-1
rpmlib(PayloadFilesHavePrefix) <= 4.0-1
rtld(GNU_HASH)
The fuse driver is loaded and the device node is there:
# ls -la dev/fuse
crw------- 1 root root 10, 229 Nov 22 22:59 dev/fuse
There are no errors reported on the console.
Here is what the debug level log reports:
# cat glusterfs.log
2008-11-23 01:34:16 D [glusterfs.c:167:get_spec_fp] glusterfs: loading
spec from /etc/glusterfs/root.vol
2008-11-23 01:34:16 D [spec.y:107:new_section] parser: New node for 'root2'
2008-11-23 01:34:16 D [xlator.c:115:xlator_set_type] xlator: attempt to
load file /usr/lib64/glusterfs/1.3.12/xlator/protocol/client.so
2008-11-23 01:34:16 D [spec.y:127:section_type] parser:
Type:root2:protocol/client
2008-11-23 01:34:16 D [spec.y:141:section_option] parser:
Option:root2:transport-type:tcp/client
2008-11-23 01:34:16 D [spec.y:141:section_option] parser:
Option:root2:remote-host:192.168.10.2
2008-11-23 01:34:16 D [spec.y:141:section_option] parser:
Option:root2:remote-subvolume:root2
2008-11-23 01:34:16 D [spec.y:209:section_end] parser: end:root2
2008-11-23 01:34:16 D [spec.y:107:new_section] parser: New node for
'root-store'
2008-11-23 01:34:16 D [xlator.c:115:xlator_set_type] xlator: attempt to
load file /usr/lib64/glusterfs/1.3.12/xlator/storage/posix.so
2008-11-23 01:34:16 D [spec.y:127:section_type] parser:
Type:root-store:storage/posix
2008-11-23 01:34:16 D [spec.y:141:section_option] parser:
Option:root-store:directory:/mnt/tmproot/gluster/root
2008-11-23 01:34:16 D [spec.y:209:section_end] parser: end:root-store
2008-11-23 01:34:16 D [spec.y:107:new_section] parser: New node for 'root1'
2008-11-23 01:34:16 D [xlator.c:115:xlator_set_type] xlator: attempt to
load file /usr/lib64/glusterfs/1.3.12/xlator/features/posix-locks.so
2008-11-23 01:34:16 D [xlator.c:145:xlator_set_type] xlator:
dlsym(notify) on
/usr/lib64/glusterfs/1.3.12/xlator/features/posix-locks.so: undefined
symbol: notify -- neglecting
2008-11-23 01:34:16 D [spec.y:127:section_type] parser:
Type:root1:features/posix-locks
2008-11-23 01:34:16 D [spec.y:196:section_sub] parser:
child:root1->root-store
2008-11-23 01:34:16 D [spec.y:209:section_end] parser: end:root1
2008-11-23 01:34:16 D [spec.y:107:new_section] parser: New node for 'server'
2008-11-23 01:34:16 D [xlator.c:115:xlator_set_type] xlator: attempt to
load file /usr/lib64/glusterfs/1.3.12/xlator/protocol/server.so
2008-11-23 01:34:16 D [spec.y:127:section_type] parser:
Type:server:protocol/server
2008-11-23 01:34:16 D [spec.y:141:section_option] parser:
Option:server:transport-type:tcp/server
2008-11-23 01:34:16 D [spec.y:196:section_sub] parser: child:server->root1
2008-11-23 01:34:16 D [spec.y:141:section_option] parser:
Option:server:auth.ip.root1.allow:127.0.0.1,192.168.*
2008-11-23 01:34:16 D [spec.y:209:section_end] parser: end:server
2008-11-23 01:34:16 D [spec.y:107:new_section] parser: New node for 'root'
2008-11-23 01:34:16 D [xlator.c:115:xlator_set_type] xlator: attempt to
load file /usr/lib64/glusterfs/1.3.12/xlator/cluster/afr.so
2008-11-23 01:34:16 D [spec.y:127:section_type] parser:
Type:root:cluster/afr
2008-11-23 01:34:16 D [spec.y:196:section_sub] parser: child:root->root1
2008-11-23 01:34:16 D [spec.y:196:section_sub] parser: child:root->root2
2008-11-23 01:34:16 D [spec.y:141:section_option] parser:
Option:root:read-subvolume:root1
2008-11-23 01:34:16 D [spec.y:209:section_end] parser: end:root
2008-11-23 01:34:16 D [glusterfs.c:128:fuse_graph] glusterfs: setting
option mount-point to /mnt/newroot
2008-11-23 01:34:16 D [xlator.c:115:xlator_set_type] xlator: attempt to
load file /usr/lib64/glusterfs/1.3.12/xlator/mount/fuse.so
2008-11-23 01:34:16 E [fuse-bridge.c:2699:init] glusterfs-fuse:
fuse_mount failed (Inappropriate ioctl for device)
2008-11-23 01:34:16 E [glusterfs.c:547:main] glusterfs: Initializing
FUSE failed
And here is the root volume spec:
# cat /etc/glusterfs/root.vol
volume root2
type protocol/client
option transport-type tcp/client
#option transport-type socket
#option address-family inet
option remote-host 192.168.10.2
option remote-subvolume root2
end-volume
volume root-store
type storage/posix
option directory /mnt/tmproot/gluster/root
end-volume
volume root1
type features/posix-locks
subvolumes root-store
end-volume
volume server
type protocol/server
option transport-type tcp/server
#option transport-type socket
#option address-family inet
subvolumes root1
option auth.ip.root1.allow 127.0.0.1,192.168.*
#option auth.addr.root1.allow 127.0.0.1,192.168.*
end-volume
volume root
type cluster/afr
subvolumes root1 root2
option read-subvolume root1
end-volume
Any suggestions on where I should even begin to look for the cause of
this problem? I tested it in single-user mode, and it seems to mount OK
there, so it has that feel of a missing dependency to it, but I can't
think of what it could be.
I can provide an image of the kernel/initrd I'm using which could be
dropped into a virtual machine for testing, if it'd help (I'm testing it
on a VM myself).
Many thanks.
Gordan
