Hi, I'm trying to use unify to replace my NFS servers but i have some problems. In my tests I also see about a quarter of the fiels that I see on NFS. I also get the following errors on my glusterfsd.log: Version : glusterfs 2.0.0rc1 built on Jan 15 2009 00:02:28 TLA Revision : glusterfs--mainline--3.0--patch-844 Starting Time: 2009-01-15 13:50:01 Command line : glusterfsd given volfile +----- 1: volume disk 2: type storage/posix 3: option directory /homes/davinci 4: end-volume 5: 6: volume disk-rs 7: type features/filter 8: option root-squashing enable 9: subvolumes disk 10: end-volume 11: 12: volume iot 13: type performance/io-threads 14: subvolumes disk-rs 15: option thread-count 4 16: end-volume 17: 18: volume brick 19: type performance/write-behind 20: subvolumes iot 21: option window-size 2MB 22: option aggregate-size 1MB 23: end-volume 24: 25: # Volume name is server 26: volume server 27: type protocol/server 28: option transport-type tcp 29: option auth.addr.brick.allow * 30: subvolumes brick 31: end-volume +----- 2009-01-15 13:50:01 W [xlator.c:382:validate_xlator_volume_options] brick: option 'aggregate-size' i s deprecated, preferred is 'block-size', continuing with correction 2009-01-15 13:50:01 W [xlator.c:382:validate_xlator_volume_options] brick: option 'window-size' is d eprecated, preferred is 'cache-size', continuing with correction 2009-01-15 14:01:42 E [socket.c:104:__socket_rwv] server: readv failed (Connection reset by peer) 2009-01-15 14:01:42 E [socket.c:566:socket_proto_state_machine] server: socket read failed (Connecti on reset by peer) in state 1 (192.168.1.235:1020) 2009-01-15 14:04:07 W [posix.c:1042:posix_link] disk: link /filipe/.Xauthority-n to /filipe/.Xauthor ity failed: File exists 2009-01-15 14:04:08 W [posix.c:796:posix_mkdir] disk: mkdir of /filipe/.dbus: File exists 2009-01-15 14:04:08 W [posix.c:796:posix_mkdir] disk: mkdir of /filipe/.dbus/session-bus: File exist s 2009-01-15 14:04:08 W [posix.c:796:posix_mkdir] disk: mkdir of /filipe/GNUstep: File exists 2009-01-15 14:04:08 W [posix.c:796:posix_mkdir] disk: mkdir of /filipe/GNUstep/Defaults: File exists 2009-01-15 14:04:08 W [posix.c:796:posix_mkdir] disk: mkdir of /filipe/GNUstep/Library: File exists 2009-01-15 14:04:08 W [posix.c:796:posix_mkdir] disk: mkdir of /filipe/GNUstep/Library/WindowMaker: File exists 2009-01-15 14:04:08 W [posix.c:796:posix_mkdir] disk: mkdir of /filipe/GNUstep/Library/WindowMaker/B ackgrounds: File exists 2009-01-15 14:04:08 W [posix.c:796:posix_mkdir] disk: mkdir of /filipe/GNUstep/Library/WindowMaker/I conSets: File exists 2009-01-15 14:04:08 W [posix.c:796:posix_mkdir] disk: mkdir of /filipe/GNUstep/Library/WindowMaker/S oundSets: File exists 2009-01-15 14:04:08 W [posix.c:796:posix_mkdir] disk: mkdir of /filipe/GNUstep/Library/WindowMaker/P ixmaps: File exists 2009-01-15 14:04:08 W [posix.c:796:posix_mkdir] disk: mkdir of /filipe/GNUstep/Library/Icons: File e xists 2009-01-15 14:04:08 W [posix.c:796:posix_mkdir] disk: mkdir of /filipe/GNUstep/Library/WindowMaker/S ounds: File exists 2009-01-15 14:04:08 W [posix.c:796:posix_mkdir] disk: mkdir of /filipe/GNUstep/Library/WindowMaker/S tyles: File exists 2009-01-15 14:04:08 W [posix.c:796:posix_mkdir] disk: mkdir of /filipe/GNUstep/Library/WindowMaker/T hemes: File exists 2009-01-15 14:08:57 E [socket.c:104:__socket_rwv] server: readv failed (Connection reset by peer) 2009-01-15 14:08:57 E [socket.c:566:socket_proto_state_machine] server: socket read failed (Connecti on reset by peer) in state 1 (192.168.1.235:1019) 2009-01-15 15:34:44 W [posix.c:796:posix_mkdir] disk: mkdir of /filipe/.kde: File exists 2009-01-15 15:34:46 W [posix.c:796:posix_mkdir] disk: mkdir of /filipe/.kde/share: File exists 2009-01-15 15:34:46 W [posix.c:796:posix_mkdir] disk: mkdir of /filipe/.kde/share/config: File exist s 2009-01-15 15:34:46 W [posix.c:928:posix_symlink] disk: symlink of /filipe/.kde/socket-gauguin --> / tmp/ksocket-filipe: File exists 2009-01-15 15:34:46 W [posix.c:796:posix_mkdir] disk: mkdir of /filipe/.qt: File exists 2009-01-15 15:34:47 W [posix.c:796:posix_mkdir] disk: mkdir of /filipe/.kde/share/apps: File exists 2009-01-15 15:35:02 W [posix.c:796:posix_mkdir] disk: mkdir of /filipe/.mcop: File exists 2009-01-15 15:35:57 W [posix.c:796:posix_mkdir] disk: mkdir of /filipe/.ssh: File exists 2009-01-15 15:37:58 E [socket.c:104:__socket_rwv] server: readv failed (Connection reset by peer) 2009-01-15 15:37:58 E [socket.c:566:socket_proto_state_machine] server: socket read failed (Connecti on reset by peer) in state 1 (192.168.1.235:1016) 2009-01-15 16:04:32 E [socket.c:104:__socket_rwv] server: writev failed (Connection reset by peer) 2009-01-15 16:05:16 E [write-behind.c:1150:wb_flush] brick: returning EBADFD 2009-01-15 16:05:17 E [write-behind.c:1150:wb_flush] brick: returning EBADFD 2009-01-15 16:05:17 E [write-behind.c:1150:wb_flush] brick: returning EBADFD 2009-01-15 16:05:17 E [write-behind.c:1150:wb_flush] brick: returning EBADFD 2009-01-15 16:05:17 E [write-behind.c:1150:wb_flush] brick: returning EBADFD 2009-01-15 16:05:17 E [write-behind.c:1150:wb_flush] brick: returning EBADFD 2009-01-15 16:05:17 E [write-behind.c:1150:wb_flush] brick: returning EBADFD 2009-01-15 16:05:17 E [write-behind.c:1150:wb_flush] brick: returning EBADFD I don't think I have any hardware problems as I can cat all the files in my home directory without any problem. I tried to reproduce the problem with a smaller setup without much luck unfortunately. Here is the client file: volume tintoretto type protocol/client option transport-type tcp option remote-host tintoretto option remote-subvolume brick end-volume volume giotto type protocol/client option transport-type tcp option remote-host giotto option remote-subvolume brick end-volume volume michelangelo type protocol/client option transport-type tcp option remote-host michelangelo option remote-subvolume brick volume donatello type protocol/client option transport-type tcp option remote-host donatello option remote-subvolume brick end-volume volume ns type protocol/client option transport-type tcp option remote-host tintoretto option remote-subvolume ns end-volume volume bricks type cluster/unify option namespace ns # this will not be storage child of unify. subvolumes tintoretto michelangelo giotto donatello # option self-heal foreground # foreground off # default is foreground # option self-heal background # foreground off # default is foreground ### ** Round Robin (RR) Scheduler ** option scheduler rr # A server is not used if it's free disk space drops below 15%. option scheduler.limits.min-free-disk 15% #% end-volume volume bricks-rs type features/filter option root-squashing enable subvolumes bricks end-volume volume iot type performance/io-threads subvolumes bricks-rs option thread-count 4 end-volume volume wb type performance/write-behind subvolumes iot option flush-behind off # default value is 'off' option window-size 2MB option aggregate-size 1MB # default value is 0 end-volume ### 'IO-Cache' translator is best used on client side when a filesystem has file # which are not modified frequently but read several times. For example, while # compiling a kernel, *.h files are read while compiling every *.c file, in these # case, io-cache translator comes very handy, as it keeps the whole file content in # the cache, and serves from the cache. # One can provide the priority of the cache too. volume ioc type performance/io-cache subvolumes wb option page-size 1MB # 128KB is default option cache-size 64MB # 32MB is default option cache-timeout 5 # 1second is default option priority *.c:2,*.h:1 # default is *:0 end-volume ### 'Read-Ahead' translator is best utilized on client side, as it prefetches # the file contents when the first read() call is issued. volume ra type performance/read-ahead subvolumes ioc option page-size 1MB # default is 256KB option page-count 4 # default is 2 option force-atime-update no # defalut is 'no' end-volume Filipe
