It's reproducable. As soon as I try to read the GlusterFS mounted share I get the error. GCC information (that's there where I run the client and the server. On the other server I use gcc 3.4.6): -------------------- nemesis ~ # gcc -v Using built-in specs. Target: i686-pc-linux-gnu Configured with: /var/tmp/portage/sys-devel/gcc-4.3.2/work/gcc-4.3.2/configure --prefix=/usr --bindir=/usr/i686-pc-linux-gnu/gcc-bin/4.3.2 --includedir=/usr/lib/gcc/i686-pc-linux-gnu/4.3.2/include --datadir=/usr/share/gcc-data/i686-pc-linux-gnu/4.3.2 --mandir=/usr/share/gcc-data/i686-pc-linux-gnu/4.3.2/man --infodir=/usr/share/gcc-data/i686-pc-linux-gnu/4.3.2/info --with-gxx-include-dir=/usr/lib/gcc/i686-pc-linux-gnu/4.3.2/include/g++-v4 --host=i686-pc-linux-gnu --build=i686-pc-linux-gnu --disable-altivec --enable-nls --without-included-gettext --with-system-zlib --disable-checking --disable-werror --enable-secureplt --disable-multilib --enable-libmudflap --disable-libssp --enable-cld --disable-libgcj --with-arch=i686 --enable-languages=c,c++,treelang --enable-shared --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu --with-bugurl=http://bugs.gentoo.org/ --with-pkgversion='Gentoo 4.3.2 p1.0' Thread model: posix gcc version 4.3.2 (Gentoo 4.3.2 p1.0) nemesis ~ # -------------------- Starting SERVER: -------------------- 2008-11-06 15:38:58 D [glusterfs.c:291:_get_specfp] glusterfs: loading volume specfile /etc/glusterfs/glusterfs-server.vol Version : glusterfs 1.4.0pre7 built on Nov 6 2008 14:23:21 TLA Revision : glusterfs--mainline--3.0--patch-561 Starting Time: 2008-11-06 15:38:58 Command line : /usr/sbin/glusterfsd -N -l/dev/stdout -L DEBUG -f /etc/glusterfs/glusterfs-server.vol given volume specfile +----- 1: ############################################## 2: ### GlusterFS Server Volume Specification ## 3: ### NEMESIS ## 4: ############################################## 5: 6: # dataspace on local 7: volume gfs-ds 8: type storage/posix # POSIX FS translator 9: option directory /local/gfs-brick001 # Export this directoy 10: end-volume 11: 12: # posix locks on local 13: volume gfs-ds-locks 14: type features/posix-locks 15: subvolumes gfs-ds 16: option mandatory on # Enables mandatory locking on all files 17: end-volume 18: 19: # dataspace on remote 20: volume gfs-remote-ds 21: type protocol/client 22: option transport-type tcp/client # For TCP/IP transport 23: option remote-host 192.168.0.115 # IP address of the remote storage 24: option remote-subvolume gfs-ds-locks 25: option transport-timeout 10 # Value in seconds; it should be set relatively low 26: end-volume 27: 28: # automatic file replication translator for dataspace 29: #volume gfs-ds-afr 30: volume gfs 31: type cluster/afr 32: subvolumes gfs-ds-locks gfs-remote-ds # Local and remote dataspaces 33: end-volume 34: 35: # the actual exported volume 36: #volume gfs 37: # type performance/io-threads 38: # option thread-count 8 # Deault is 1 39: # option cache-size 64MB # Default is 64MB 40: # subvolumes gfs-ds-afr 41: #end-volume 42: 43: # server declaration 44: volume server 45: type protocol/server 46: option transport-type tcp/server # For TCP/IP transport 47: subvolumes gfs 48: # storage network access only 49: option auth.addr.gfs-ds-locks.allow 192.168.0.*,127.0.0.1 50: option auth.addr.gfs.allow 192.168.0.* 51: end-volume +----- 2008-11-06 15:38:58 D [spec.y:178:new_section] parser: New node for 'gfs-ds' 2008-11-06 15:38:58 D [xlator.c:367:xlator_set_type] xlator: attempt to load file /usr/lib/glusterfs/1.4.0pre7/xlator/storage/posix.so 2008-11-06 15:38:58 D [spec.y:202:section_type] parser: Type:gfs-ds:storage/posix 2008-11-06 15:38:58 D [spec.y:268:section_option] parser: Option:gfs-ds:directory:/local/gfs-brick001 2008-11-06 15:38:58 D [spec.y:350:section_end] parser: end:gfs-ds 2008-11-06 15:38:58 D [spec.y:178:new_section] parser: New node for 'gfs-ds-locks' 2008-11-06 15:38:58 D [xlator.c:367:xlator_set_type] xlator: attempt to load file /usr/lib/glusterfs/1.4.0pre7/xlator/features/posix-locks.so 2008-11-06 15:38:58 D [xlator.c:407:xlator_set_type] xlator: dlsym(notify) on /usr/lib/glusterfs/1.4.0pre7/xlator/features/posix-locks.so: undefined symbol: notify -- neglecting 2008-11-06 15:38:58 D [spec.y:202:section_type] parser: Type:gfs-ds-locks:features/posix-locks 2008-11-06 15:38:58 D [spec.y:335:section_sub] parser: child:gfs-ds-locks->gfs-ds 2008-11-06 15:38:58 D [spec.y:268:section_option] parser: Option:gfs-ds-locks:mandatory:on 2008-11-06 15:38:58 D [spec.y:350:section_end] parser: end:gfs-ds-locks 2008-11-06 15:38:58 D [spec.y:178:new_section] parser: New node for 'gfs-remote-ds' 2008-11-06 15:38:58 D [xlator.c:367:xlator_set_type] xlator: attempt to load file /usr/lib/glusterfs/1.4.0pre7/xlator/protocol/client.so 2008-11-06 15:38:58 D [spec.y:202:section_type] parser: Type:gfs-remote-ds:protocol/client 2008-11-06 15:38:58 D [spec.y:268:section_option] parser: Option:gfs-remote-ds:transport-type:tcp/client 2008-11-06 15:38:58 D [spec.y:268:section_option] parser: Option:gfs-remote-ds:remote-host:192.168.0.115 2008-11-06 15:38:58 D [spec.y:268:section_option] parser: Option:gfs-remote-ds:remote-subvolume:gfs-ds-locks 2008-11-06 15:38:58 D [spec.y:268:section_option] parser: Option:gfs-remote-ds:transport-timeout:10 2008-11-06 15:38:58 D [spec.y:350:section_end] parser: end:gfs-remote-ds 2008-11-06 15:38:58 D [spec.y:178:new_section] parser: New node for 'gfs' 2008-11-06 15:38:58 D [xlator.c:367:xlator_set_type] xlator: attempt to load file /usr/lib/glusterfs/1.4.0pre7/xlator/cluster/afr.so 2008-11-06 15:38:58 D [spec.y:202:section_type] parser: Type:gfs:cluster/afr 2008-11-06 15:38:58 D [spec.y:335:section_sub] parser: child:gfs->gfs-ds-locks 2008-11-06 15:38:58 D [spec.y:335:section_sub] parser: child:gfs->gfs-remote-ds 2008-11-06 15:38:58 D [spec.y:350:section_end] parser: end:gfs 2008-11-06 15:38:58 D [spec.y:178:new_section] parser: New node for 'server' 2008-11-06 15:38:58 D [xlator.c:367:xlator_set_type] xlator: attempt to load file /usr/lib/glusterfs/1.4.0pre7/xlator/protocol/server.so 2008-11-06 15:38:58 D [spec.y:202:section_type] parser: Type:server:protocol/server 2008-11-06 15:38:58 D [spec.y:268:section_option] parser: Option:server:transport-type:tcp/server 2008-11-06 15:38:58 D [spec.y:335:section_sub] parser: child:server->gfs 2008-11-06 15:38:58 D [spec.y:268:section_option] parser: Option:server:auth.addr.gfs-ds-locks.allow:192.168.0.*,127.0.0.1 2008-11-06 15:38:58 D [spec.y:268:section_option] parser: Option:server:auth.addr.gfs.allow:192.168.0.* 2008-11-06 15:38:58 D [spec.y:350:section_end] parser: end:server 2008-11-06 15:38:58 D [glusterfs.c:804:main] glusterfs: running in pid 24152 2008-11-06 15:38:58 D [transport.c:104:transport_load] transport: attempt to load file /usr/lib/glusterfs/1.4.0pre7/transport/socket.so 2008-11-06 15:38:58 D [server-protocol.c:7130:init] server: defaulting limits.transaction-size to 4194304 2008-11-06 15:38:58 D [xlator.c:491:xlator_init_rec] gfs-ds: Initialization done 2008-11-06 15:38:58 D [xlator.c:491:xlator_init_rec] gfs-ds-locks: Initialization done 2008-11-06 15:38:58 D [client-protocol.c:4918:init] gfs-remote-ds: setting transport-timeout to 10 2008-11-06 15:38:58 D [transport.c:104:transport_load] transport: attempt to load file /usr/lib/glusterfs/1.4.0pre7/transport/socket.so 2008-11-06 15:38:58 D [client-protocol.c:4962:init] gfs-remote-ds: defaulting limits.transaction-size to 268435456 2008-11-06 15:38:58 D [xlator.c:491:xlator_init_rec] gfs-remote-ds: Initialization done 2008-11-06 15:38:58 D [client-protocol.c:5194:notify] gfs-remote-ds: got GF_EVENT_PARENT_UP, attempting connect on transport 2008-11-06 15:38:58 D [client-protocol.c:5194:notify] gfs-remote-ds: got GF_EVENT_PARENT_UP, attempting connect on transport 2008-11-06 15:38:58 D [client-protocol.c:4610:client_protocol_reconnect] gfs-remote-ds: attempting reconnect 2008-11-06 15:38:58 D [name.c:182:af_inet_client_get_remote_sockaddr] gfs-remote-ds: option remote-port missing in volume gfs-remote-ds. Defaulting to 6996 2008-11-06 15:38:58 D [common-utils.c:213:gf_resolve_ip6] resolver: DNS cache not present, freshly probing hostname: 192.168.0.115 2008-11-06 15:38:58 D [common-utils.c:250:gf_resolve_ip6] resolver: returning ip-192.168.0.115 (port-6996) for hostname: 192.168.0.115 and port: 6996 2008-11-06 15:38:58 D [client-protocol.c:5231:notify] gfs-remote-ds: got GF_EVENT_CHILD_UP 2008-11-06 15:38:58 D [socket.c:924:socket_connect] gfs-remote-ds: connect () called on transport already connected 2008-11-06 15:38:58 D [client-protocol.c:4549:client_setvolume_cbk] gfs-remote-ds: SETVOLUME on remote-host succeeded 2008-11-06 15:38:59 D [client-protocol.c:4616:client_protocol_reconnect] gfs-remote-ds: breaking reconnect chain 2008-11-06 15:39:09 D [addr.c:166:gf_auth] gfs: allowed = "192.168.0.*", received addr = "192.168.0.145" 2008-11-06 15:39:09 D [server-protocol.c:6406:mop_setvolume] server: accepted client from 192.168.0.145:1022 2008-11-06 15:39:09 D [server-protocol.c:6440:mop_setvolume] server: creating inode table with lru_limit=1024, xlator=gfs 2008-11-06 15:39:09 D [inode.c:934:inode_table_new] gfs: creating new inode table with lru_limit=1024 2008-11-06 15:39:09 D [inode.c:443:__inode_create] gfs/inode: create inode(0) 2008-11-06 15:39:19 D [inode.c:268:__inode_activate] gfs/inode: activating inode(1), lru=0/1024 active=1 purge=0 2008-11-06 15:39:19 D [afr.c:335:afr_lookup_cbk] gfs: scaling inode 1 to 3 *** glibc detected *** /usr/sbin/glusterfsd: free(): invalid pointer: 0x08059f4e *** -------------------- Starting client: -------------------- 2008-11-06 15:39:09 D [glusterfs.c:291:_get_specfp] glusterfs: loading volume specfile /etc/glusterfs/glusterfs-client.vol Version : glusterfs 1.4.0pre7 built on Nov 6 2008 14:23:21 TLA Revision : glusterfs--mainline--3.0--patch-561 Starting Time: 2008-11-06 15:39:09 Command line : /usr/sbin/glusterfs -N -l/dev/stdout -L DEBUG -f /etc/glusterfs/glusterfs-client.vol /home/vmail/ given volume specfile +----- 1: ############################################# 2: ## GlusterFS Client Volume Specification ## 3: ############################################# 4: 5: # the exported volume to mount 6: volume cluster 7: type protocol/client 8: option transport-type tcp/client 9: option remote-host gfs-vmail001.vunet.local # RRDNS 10: option remote-subvolume gfs # Exported volume 11: option transport-timeout 10 # Value in seconds, should be relatively low 12: end-volume 13: 14: # performance block for cluster (Write Behind Translator) 15: #volume writeback 16: # type performance/write-behind 17: # option aggregate-size 1MB # Default is 0bytes 18: # option window-size 3MB # Default is 0bytes 19: # option flush-behind on # Default is 'off' 20: # subvolumes cluster 21: #end-volume 22: 23: # performance block for cluster (Read Ahead Translator) 24: #volume readahead 25: # type performance/read-ahead 26: # option page-size 65KB # 256KB is the default option 27: # option page-count 16 # 2 is default option 28: # option force-atime-update off # Default is off 29: # subvolumes writeback 30: #end-volume +----- 2008-11-06 15:39:09 D [spec.y:178:new_section] parser: New node for 'cluster' 2008-11-06 15:39:09 D [xlator.c:367:xlator_set_type] xlator: attempt to load file /usr/lib/glusterfs/1.4.0pre7/xlator/protocol/client.so 2008-11-06 15:39:09 D [spec.y:202:section_type] parser: Type:cluster:protocol/client 2008-11-06 15:39:09 D [spec.y:268:section_option] parser: Option:cluster:transport-type:tcp/client 2008-11-06 15:39:09 D [spec.y:268:section_option] parser: Option:cluster:remote-host:gfs-vmail001.vunet.local 2008-11-06 15:39:09 D [spec.y:268:section_option] parser: Option:cluster:remote-subvolume:gfs 2008-11-06 15:39:09 D [spec.y:268:section_option] parser: Option:cluster:transport-timeout:10 2008-11-06 15:39:09 D [spec.y:350:section_end] parser: end:cluster 2008-11-06 15:39:09 D [xlator.c:367:xlator_set_type] xlator: attempt to load file /usr/lib/glusterfs/1.4.0pre7/xlator/mount/fuse.so 2008-11-06 15:39:09 D [glusterfs.c:804:main] glusterfs: running in pid 24155 2008-11-06 15:39:09 D [fuse-options.c:149:fuse_options_validate] fuse-options: using mount-point = /home/vmail/ 2008-11-06 15:39:09 D [fuse-options.c:156:fuse_options_validate] fuse-options: using attr-timeout = 1 2008-11-06 15:39:09 D [fuse-options.c:168:fuse_options_validate] fuse-options: using entry-timeout = 1 2008-11-06 15:39:09 D [fuse-options.c:180:fuse_options_validate] fuse-options: using direct-io-mode = 1 2008-11-06 15:39:09 D [client-protocol.c:4918:init] cluster: setting transport-timeout to 10 2008-11-06 15:39:09 D [transport.c:104:transport_load] transport: attempt to load file /usr/lib/glusterfs/1.4.0pre7/transport/socket.so 2008-11-06 15:39:09 D [client-protocol.c:4962:init] cluster: defaulting limits.transaction-size to 268435456 2008-11-06 15:39:09 D [client-protocol.c:5194:notify] cluster: got GF_EVENT_PARENT_UP, attempting connect on transport 2008-11-06 15:39:09 D [inode.c:934:inode_table_new] fuse: creating new inode table with lru_limit=0 2008-11-06 15:39:09 D [inode.c:443:__inode_create] fuse/inode: create inode(0) 2008-11-06 15:39:09 D [client-protocol.c:5194:notify] cluster: got GF_EVENT_PARENT_UP, attempting connect on transport 2008-11-06 15:39:09 D [client-protocol.c:4610:client_protocol_reconnect] cluster: attempting reconnect 2008-11-06 15:39:09 D [name.c:182:af_inet_client_get_remote_sockaddr] cluster: option remote-port missing in volume cluster. Defaulting to 6996 2008-11-06 15:39:09 D [common-utils.c:213:gf_resolve_ip6] resolver: DNS cache not present, freshly probing hostname: gfs-vmail001.vunet.local 2008-11-06 15:39:09 D [common-utils.c:250:gf_resolve_ip6] resolver: returning ip-192.168.0.145 (port-6996) for hostname: gfs-vmail001.vunet.local and port: 6996 2008-11-06 15:39:09 D [common-utils.c:270:gf_resolve_ip6] resolver: next DNS query will return: ip-192.168.0.115 port-6996 2008-11-06 15:39:09 D [client-protocol.c:5231:notify] cluster: got GF_EVENT_CHILD_UP 2008-11-06 15:39:09 D [socket.c:924:socket_connect] cluster: connect () called on transport already connected 2008-11-06 15:39:09 D [client-protocol.c:4549:client_setvolume_cbk] cluster: SETVOLUME on remote-host succeeded 2008-11-06 15:39:10 D [client-protocol.c:4616:client_protocol_reconnect] cluster: breaking reconnect chain 2008-11-06 15:39:19 D [inode.c:268:__inode_activate] fuse/inode: activating inode(1), lru=0/0 active=1 purge=0 2008-11-06 15:39:19 D [fuse-bridge.c:334:fuse_entry_cbk] glusterfs-fuse: 2: LOOKUP() / => 1 2008-11-06 15:39:19 D [fuse-bridge.c:2137:fuse_getxattr] glusterfs-fuse: 3: GETXATTR //1 (system.posix_acl_access) 2008-11-06 15:39:19 D [fuse-bridge.c:1987:fuse_xattr_cbk] glusterfs-fuse: 3: GETXATTR() / => 50 2008-11-06 15:39:19 D [fuse-bridge.c:2137:fuse_getxattr] glusterfs-fuse: 4: GETXATTR //1 (system.posix_acl_default) 2008-11-06 15:39:39 E [client-protocol.c:240:call_bail] cluster: activating bail-out. pending frames = 1. last sent = 2008-11-06 15:39:19. last received = 2008-11-06 15:39:19. transport-timeout = 10 2008-11-06 15:39:39 C [client-protocol.c:247:call_bail] cluster: bailing transport 2008-11-06 15:39:39 D [socket.c:183:__socket_disconnect] cluster: shutdown() returned 0. setting connection state to -1 2008-11-06 15:39:39 D [socket.c:93:__socket_rwv] cluster: EOF from peer 192.168.0.145:6996 2008-11-06 15:39:39 D [socket.c:568:socket_proto_state_machine] cluster: socket read failed (Transport endpoint is not connected) in state 1 (192.168.0.145:6996) 2008-11-06 15:39:39 D [client-protocol.c:4636:protocol_client_cleanup] cluster: cleaning up state in transport object 0x8056218 2008-11-06 15:39:39 E [client-protocol.c:4691:protocol_client_cleanup] cluster: forced unwinding frame type(1) op(GETXATTR) reply=@0x8090800 2008-11-06 15:39:39 E [fuse-bridge.c:2093:fuse_xattr_cbk] glusterfs-fuse: 4: GETXATTR() / => -1 (Transport endpoint is not connected) 2008-11-06 15:39:39 E [socket.c:1187:socket_submit] cluster: transport not connected to submit (priv->connected = 255) 2008-11-06 15:39:39 D [inode.c:443:__inode_create] fuse/inode: create inode(0) 2008-11-06 15:39:39 D [inode.c:268:__inode_activate] fuse/inode: activating inode(0), lru=0/0 active=2 purge=0 2008-11-06 15:39:39 E [fuse-bridge.c:364:fuse_entry_cbk] glusterfs-fuse: 5: LOOKUP() / => -1 (Transport endpoint is not connected) 2008-11-06 15:39:39 D [inode.c:311:__inode_retire] fuse/inode: retiring inode(0) lru=0/0 active=1 purge=1 2008-11-06 15:39:39 D [client-protocol.c:4610:client_protocol_reconnect] cluster: attempting reconnect 2008-11-06 15:39:39 D [name.c:182:af_inet_client_get_remote_sockaddr] cluster: option remote-port missing in volume cluster. Defaulting to 6996 2008-11-06 15:39:39 D [common-utils.c:250:gf_resolve_ip6] resolver: returning ip-192.168.0.115 (port-6996) for hostname: gfs-vmail001.vunet.local and port: 6996 2008-11-06 15:39:39 D [client-protocol.c:5231:notify] cluster: got GF_EVENT_CHILD_UP 2008-11-06 15:39:39 D [socket.c:924:socket_connect] cluster: connect () called on transport already connected 2008-11-06 15:39:39 D [client-protocol.c:4549:client_setvolume_cbk] cluster: SETVOLUME on remote-host succeeded 2008-11-06 15:39:40 D [client-protocol.c:4616:client_protocol_reconnect] cluster: breaking reconnect chain 2008-11-06 15:40:42 W [glusterfs.c:548:cleanup_and_exit] glusterfs: shutting down 2008-11-06 15:40:42 W [fuse-bridge.c:2685:fini] fuse: unmounting /home/vmail/ 2008-11-06 15:40:42 W [glusterfs.c:548:cleanup_and_exit] glusterfs: shutting down 2008-11-06 15:40:42 D [glusterfs.c:569:cleanup_and_exit] glusterfs: no graph present 2008-11-06 15:40:42 D [dict.c:353:dict_destroy] dict: @this=(nil) 2008-11-06 15:40:42 D [dict.c:353:dict_destroy] dict: @this=(nil) -------------------- I used FUSE 2.8.0_pre1 but switched back to FUSE 2.7.4. But this did not help. Same error even after recompiling GlusterFS after the fresh install of FUSE 2.7.4. I did not have those issues (the crashing stuff) with GlusterFS 1.3.x. I had other issues (with locking files etc) and that is the reason I started to look at the TLA version since I have read in a bug report that a certain patchlevel fixes that particular issue with locking. Well... and now I have other issues and my old issue is still not fixed. Let me know if you need more info from me. // Steve -------- Original-Nachricht -------- > Datum: Thu, 6 Nov 2008 19:22:44 +0530 > Von: "Vikas Gorur" <vikas@xxxxxxxxxxxxx> > An: Steve <steeeeeveee@xxxxxxx> > CC: gluster-devel@xxxxxxxxxx > Betreff: Re: glusterfs--mainline--3.0 patch-556 > 2008/11/6 Steve <steeeeeveee@xxxxxxx>: > > Hallo Raghavendra > > > > Now I get this, wenn I try to read a GlusterFS mounted filesystem: > > ---- > > 2008-11-06 14:13:31 D [inode.c:268:__inode_activate] gfs/inode: > activating inode(1), lru=0/1024 active=1 purge=0 > > 2008-11-06 14:13:31 D [afr.c:335:afr_lookup_cbk] gfs: scaling inode 1 to > 3 > > *** glibc detected *** /usr/sbin/glusterfsd: free(): invalid pointer: > 0x08091e5e *** > > ---- > > > > Do you need more information? What information? > > Hi Steve, > > Is it easy to reproduce? If so, can you give us your spec files and steps > to > reproduce it? > > If it is not reproducible, it would help us if you had a core file and > could get > a backtrace. > > Vikas > -- > Engineer - Z Research > http://gluster.com/ -- Psssst! Schon vom neuen GMX MultiMessenger gehört? Der kann`s mit allen: http://www.gmx.net/de/go/multimessenger
