> Shortest sequence of steps to reproduce?
>
Two systems in my case:
1: hermes
2: tisiphone
On both systems I do:
# /etc/init.d/glusterfs-client.vu-hosting stop
# /etc/init.d/glusterfs-server.vu-hosting stop
# /etc/init.d/fuse stop
# umount /mnt/glusterfs/vu-hosting/
# mkfs.xfs -f -l version=2 -i attr=2 -L vu-hosting /dev/LVM_RAID_VolGroup01/vu-hosting
# mount /mnt/glusterfs/vu-hosting/
# setfattr -x trusted.glusterfs.version /mnt/glusterfs/vu-hosting
# setfattr -x trusted.glusterfs.createtime /mnt/glusterfs/vu-hosting
# /etc/init.d/glusterfs-server.vu-hosting start
# /etc/init.d/glusterfs-client.vu-hosting start
Then on one of the systems I do:
# mkdir -pv /var/www/vu-hosting/localhost/{admin/ssl,htdocs,log,temp}
# cp -v /etc/ssl/nginx/nginx.* /var/www/vu-hosting/localhost/admin/ssl
# md5sum --binary /var/www/vu-hosting/localhost/admin/ssl/*
The result is:
hermes:
----
hermes ~ # /etc/init.d/glusterfs-client.vu-hosting stop ; /etc/init.d/glusterfs-server.vu-hosting stop ; /etc/init.d/fuse stop ; umount /mnt/glusterfs/vu-hosting/ ; mkfs.xfs -f -l version=2 -i attr=2 -L vu-hosting /dev/LVM_RAID_VolGroup01/vu-hosting ; mount /mnt/glusterfs/vu-hosting/ ; setfattr -x trusted.glusterfs.version /mnt/glusterfs/vu-hosting ; setfattr -x trusted.glusterfs.createtime /mnt/glusterfs/vu-hosting ; /etc/init.d/glusterfs-server.vu-hosting start ; /etc/init.d/glusterfs-client.vu-hosting start
* Stopping glusterfs-client.vu-hosting... [ ok ]
* Stopping glusterfs-server.vu-hosting... [ ok ]
* Stopping fuse... [ ok ]
meta-data=/dev/LVM_RAID_VolGroup01/vu-hosting isize=256 agcount=4, agsize=327680 blks
= sectsz=512 attr=2
data = bsize=4096 blocks=1310720, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0
log =internal log bsize=4096 blocks=2560, version=2
= sectsz=512 sunit=0 blks, lazy-count=0
realtime =none extsz=4096 blocks=0, rtextents=0
setfattr: /mnt/glusterfs/vu-hosting: No such attribute
setfattr: /mnt/glusterfs/vu-hosting: No such attribute
* Starting glusterfs-server.vu-hosting... [ ok ]
* Starting fuse... [ ok ]
* Starting glusterfs-client.vu-hosting... [ ok ]
hermes ~ #
----
tisiphone:
----
tisiphone ~ # /etc/init.d/glusterfs-client.vu-hosting stop ; /etc/init.d/glusterfs-server.vu-hosting stop ; /etc/init.d/fuse stop ; umount /mnt/glusterfs/vu-hosting/ ; mkfs.xfs -f -l version=2 -i attr=2 -L vu-hosting /dev/LVM_RAID_VolGroup01/vu-hosting ; mount /mnt/glusterfs/vu-hosting/ ; setfattr -x trusted.glusterfs.version /mnt/glusterfs/vu-hosting ; setfattr -x trusted.glusterfs.createtime /mnt/glusterfs/vu-hosting ; /etc/init.d/glusterfs-server.vu-hosting start ; /etc/init.d/glusterfs-client.vu-hosting start
* Stopping glusterfs-client.vu-hosting... [ ok ]
* Stopping glusterfs-server.vu-hosting... [ ok ]
* Stopping fuse... [ ok ]
meta-data=/dev/LVM_RAID_VolGroup01/vu-hosting isize=256 agcount=4, agsize=327680 blks
= sectsz=512 attr=2
data = bsize=4096 blocks=1310720, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0
log =internal log bsize=4096 blocks=2560, version=2
= sectsz=512 sunit=0 blks, lazy-count=0
realtime =none extsz=4096 blocks=0, rtextents=0
setfattr: /mnt/glusterfs/vu-hosting: No such attribute
setfattr: /mnt/glusterfs/vu-hosting: No such attribute
* Starting glusterfs-server.vu-hosting... [ ok ]
* Starting fuse... [ ok ]
* Starting glusterfs-client.vu-hosting... [ ok ]
tisiphone ~ #
----
Doing the copy and check job on hermes:
----
hermes ~ # mkdir -pv /var/www/vu-hosting/localhost/{admin/ssl,htdocs,log,temp} ; cp -v /etc/ssl/nginx/nginx.* /var/www/vu-hosting/localhost/admin/ssl ; echo "Source:" ; md5sum --binary /etc/ssl/nginx/nginx.* ; echo "GlusterFS:" ; md5sum --binary /var/www/vu-hosting/localhost/admin/ssl/*
mkdir: created directory `/var/www/vu-hosting/localhost'
mkdir: created directory `/var/www/vu-hosting/localhost/admin'
mkdir: created directory `/var/www/vu-hosting/localhost/admin/ssl'
mkdir: created directory `/var/www/vu-hosting/localhost/htdocs'
mkdir: created directory `/var/www/vu-hosting/localhost/log'
mkdir: created directory `/var/www/vu-hosting/localhost/temp'
`/etc/ssl/nginx/nginx.crt' -> `/var/www/vu-hosting/localhost/admin/ssl/nginx.crt'
`/etc/ssl/nginx/nginx.csr' -> `/var/www/vu-hosting/localhost/admin/ssl/nginx.csr'
`/etc/ssl/nginx/nginx.key' -> `/var/www/vu-hosting/localhost/admin/ssl/nginx.key'
`/etc/ssl/nginx/nginx.pem' -> `/var/www/vu-hosting/localhost/admin/ssl/nginx.pem'
Source:
c80162aa9220f8942fd11137f03243cb */etc/ssl/nginx/nginx.crt
330a48882caf1960180183d7e3d237c8 */etc/ssl/nginx/nginx.csr
20169d74a3493b2941c99213ea24705b */etc/ssl/nginx/nginx.key
c92bad4bdfde4df52db7afe91977247d */etc/ssl/nginx/nginx.pem
GlusterFS:
c80162aa9220f8942fd11137f03243cb */var/www/vu-hosting/localhost/admin/ssl/nginx.crt
330a48882caf1960180183d7e3d237c8 */var/www/vu-hosting/localhost/admin/ssl/nginx.csr
20169d74a3493b2941c99213ea24705b */var/www/vu-hosting/localhost/admin/ssl/nginx.key
c92bad4bdfde4df52db7afe91977247d */var/www/vu-hosting/localhost/admin/ssl/nginx.pem
hermes ~ #
----
==> All fine on hermes.
Doing the check job on tisiphone:
----
tisiphone ~ # md5sum --binary /var/www/vu-hosting/localhost/admin/ssl/*
c80162aa9220f8942fd11137f03243cb */var/www/vu-hosting/localhost/admin/ssl/nginx.crt
330a48882caf1960180183d7e3d237c8 */var/www/vu-hosting/localhost/admin/ssl/nginx.csr
20169d74a3493b2941c99213ea24705b */var/www/vu-hosting/localhost/admin/ssl/nginx.key
d41d8cd98f00b204e9800998ecf8427e */var/www/vu-hosting/localhost/admin/ssl/nginx.pem
tisiphone ~ #
----
==> The pem file is wrong!
Redoing the check on hermes:
----
hermes ~ # md5sum --binary /var/www/vu-hosting/localhost/admin/ssl/*
c80162aa9220f8942fd11137f03243cb */var/www/vu-hosting/localhost/admin/ssl/nginx.crt
330a48882caf1960180183d7e3d237c8 */var/www/vu-hosting/localhost/admin/ssl/nginx.csr
20169d74a3493b2941c99213ea24705b */var/www/vu-hosting/localhost/admin/ssl/nginx.key
c92bad4bdfde4df52db7afe91977247d */var/www/vu-hosting/localhost/admin/ssl/nginx.pem
hermes ~ #
----
Listing on hermes:
----
hermes ~ # ls -lah /var/www/vu-hosting/localhost/admin/ssl/
total 16K
drwxr-xr-x 2 root root 70 Mar 30 18:27 .
drwxr-xr-x 3 root root 16 Mar 30 18:27 ..
-r--r--r-- 1 root root 1017 Mar 30 18:27 nginx.crt
-r--r--r-- 1 root root 737 Mar 30 18:27 nginx.csr
-r-------- 1 root root 887 Mar 30 18:27 nginx.key
-r-------- 1 root root 1.9K Mar 30 18:27 nginx.pem
hermes ~ #
----
Listing on tisiphone:
----
tisiphone ~ # ls -lah /var/www/vu-hosting/localhost/admin/ssl/
total 16K
drwxr-xr-x 2 root root 70 Mar 30 18:27 .
drwxr-xr-x 3 root root 16 Mar 30 18:27 ..
-r--r--r-- 1 root root 1017 Mar 30 18:27 nginx.crt
-r--r--r-- 1 root root 737 Mar 30 18:27 nginx.csr
-r-------- 1 root root 887 Mar 30 18:27 nginx.key
-r-------- 1 root root 1.9K Mar 30 18:27 nginx.pem
tisiphone ~ #
----
==> WHAT? This is new! I never had the whole file there in my tests. Now the file is there and not having 0 bytes size. Let me check MD5 sum.
Recheck on tisiphone:
----
tisiphone ~ # md5sum --binary /var/www/vu-hosting/localhost/admin/ssl/*
c80162aa9220f8942fd11137f03243cb */var/www/vu-hosting/localhost/admin/ssl/nginx.crt
330a48882caf1960180183d7e3d237c8 */var/www/vu-hosting/localhost/admin/ssl/nginx.csr
20169d74a3493b2941c99213ea24705b */var/www/vu-hosting/localhost/admin/ssl/nginx.key
d41d8cd98f00b204e9800998ecf8427e */var/www/vu-hosting/localhost/admin/ssl/nginx.pem
tisiphone ~ #
----
==> Okay. At least this is consistent. The pem file is not the same on both nodes. The size is this time not 0 bytes but the content is not the same. At least if I can trust md5sum.
It's always the pem file which get's destroyed. I don't know why? Another case where I loose data is when I name the file "default_vhost.include". The file it self is normal ASCII but with a reference to the pem file. The effect is the same. I loose the data. The file is 0 bytes long after doing the copy to GlusterFS.
The log in NORMAL mode:
hermes (server):
----
================================================================================
Version : glusterfs 2.0.0git built on Mar 30 2009 05:38:42
TLA Revision : git://git.sv.gnu.org/gluster.git
Starting Time: 2009-03-30 18:25:31
Command line : /usr/sbin/glusterfsd -N -f /etc/glusterfs/glusterfs-server.vu-hosting.vol -l /var/log/glusterfs/glusterfs-server.vu-hosting.log -L NORMAL
PID : 6265
System name : Linux
Nodename : hermes
Kernel Release : 2.6.29
Hardware Identifier: i686
Given volfile:
+------------------------------------------------------------------------------+
1: #######################################################
2: ### GlusterFS Server Volume Specification ###
3: #######################################################
4:
5: volume gfs-ds-posix
6: type storage/posix
7: option directory /mnt/glusterfs/vu-hosting
8: # option o-direct off
9: # option export-statfs-size yes
10: # option mandate-attribute yes
11: # option span-devices 1
12: end-volume
13:
14: volume gfs-ds-locks
15: type features/locks
16: option mandatory-locks off
17: subvolumes gfs-ds-posix
18: end-volume
19:
20: volume gfs-ds-io-threads
21: type performance/io-threads
22: option thread-count 8
23: subvolumes gfs-ds-locks
24: end-volume
25:
26: volume gfs-ds-server
27: type protocol/server
28: option transport-type tcp
29: option transport.socket.listen-port 6997
30: option auth.addr.gfs-ds-locks.allow 192.168.0.*,127.0.0.1
31: option auth.addr.gfs-ds-io-threads.allow 192.168.0.*,127.0.0.1
32: subvolumes gfs-ds-io-threads
33: end-volume
+------------------------------------------------------------------------------+
2009-03-30 18:25:31 N [glusterfsd.c:1134:main] glusterfs: Successfully started
2009-03-30 18:25:31 N [server-protocol.c:7513:mop_setvolume] gfs-ds-server: accepted client from 192.168.0.70:1021
2009-03-30 18:25:31 N [server-protocol.c:7513:mop_setvolume] gfs-ds-server: accepted client from 192.168.0.70:1020
2009-03-30 18:25:33 N [server-protocol.c:7513:mop_setvolume] gfs-ds-server: accepted client from 192.168.0.75:1021
2009-03-30 18:25:33 N [server-protocol.c:7513:mop_setvolume] gfs-ds-server: accepted client from 192.168.0.75:1020
2009-03-30 18:25:37 N [server-protocol.c:8268:notify] gfs-ds-server: 192.168.0.75:1021 disconnected
2009-03-30 18:25:37 N [server-protocol.c:8268:notify] gfs-ds-server: 192.168.0.75:1020 disconnected
2009-03-30 18:25:37 N [server-helpers.c:530:server_connection_destroy] gfs-ds-server: destroyed connection of tisiphone-5818-2009/03/30-18:24:46:612808-gfs-ds-vu-hosting-hermes
2009-03-30 18:25:38 N [server-protocol.c:7513:mop_setvolume] gfs-ds-server: accepted client from 192.168.0.75:1019
2009-03-30 18:25:38 N [server-protocol.c:7513:mop_setvolume] gfs-ds-server: accepted client from 192.168.0.75:1018
2009-03-30 18:29:00 N [server-protocol.c:8268:notify] gfs-ds-server: 192.168.0.75:1019 disconnected
2009-03-30 18:29:00 N [server-protocol.c:8268:notify] gfs-ds-server: 192.168.0.75:1018 disconnected
2009-03-30 18:29:00 N [server-helpers.c:530:server_connection_destroy] gfs-ds-server: destroyed connection of tisiphone-5934-2009/03/30-18:25:41:340960-gfs-ds-vu-hosting-hermes
2009-03-30 18:29:00 N [server-protocol.c:7513:mop_setvolume] gfs-ds-server: accepted client from 192.168.0.75:1023
2009-03-30 18:29:00 N [server-protocol.c:7513:mop_setvolume] gfs-ds-server: accepted client from 192.168.0.75:1022
----
hermes (client):
----
================================================================================
Version : glusterfs 2.0.0git built on Mar 30 2009 05:38:42
TLA Revision : git://git.sv.gnu.org/gluster.git
Starting Time: 2009-03-30 18:25:31
Command line : /usr/sbin/glusterfsd -N -f /etc/glusterfs/glusterfs-client.vu-hosting.vol -l /var/log/glusterfs/glusterfs-client.vu-hosting.log -L NORMAL /var/www/vu-hosting
PID : 6300
System name : Linux
Nodename : hermes
Kernel Release : 2.6.29
Hardware Identifier: i686
Given volfile:
+------------------------------------------------------------------------------+
1: #######################################################
2: ### GlusterFS Client Volume Specification ###
3: #######################################################
4:
5: #volume gfs-ds-fuse
6: # type mount/fuse
7: # option direct-io-mode on
8: # option macfuse-local off
9: # option mount-point /var/www/vu-hosting
10: # option attribute-timeout 0.0
11: # option entry-timeout 0.0
12: #end-volume
13:
14: volume gfs-ds-vu-hosting-hermes
15: type protocol/client
16: option transport-type tcp
17: # option username
18: # option password
19: option remote-host 192.168.0.70
20: option remote-port 6997
21: option remote-subvolume gfs-ds-io-threads
22: option transport-timeout 10
23: option ping-timeout 5
24: end-volume
25:
26: volume gfs-ds-vu-hosting-tisiphone
27: type protocol/client
28: option transport-type tcp
29: # option username
30: # option password
31: option remote-host 192.168.0.75
32: option remote-port 6997
33: option remote-subvolume gfs-ds-io-threads
34: option transport-timeout 10
35: option ping-timeout 5
36: end-volume
37:
38: volume gfs-ds-replicate
39: type cluster/replicate
40: option data-self-heal on
41: option metadata-self-heal on
42: option entry-self-heal on
43: # option read-subvolume
44: # option favorite-child
45: option data-change-log on
46: option metadata-change-log on
47: option entry-change-log on
48: option data-lock-server-count 1
49: option metadata-lock-server-count 1
50: option entry-lock-server-count 1
51: subvolumes gfs-ds-vu-hosting-hermes gfs-ds-vu-hosting-tisiphone
52: end-volume
53:
54: volume gfs-ds-io-threads
55: type performance/io-threads
56: option thread-count 8
57: subvolumes gfs-ds-replicate
58: end-volume
59:
60: volume gfs-ds-write-behind
61: type performance/write-behind
62: option block-size 1MB
63: option cache-size 4MB
64: option flush-behind on
65: # opiton disable-for-first-nbytes 1
66: # option enable-O_SYNC false
67: subvolumes gfs-ds-io-threads
68: end-volume
69:
70: #volume gfs-ds-read-ahead
71: # type performance/read-ahead
72: # option page-size 65KB
73: # option page-count 16
74: # option force-atime-update off
75: # subvolumes gfs-ds-write-behind
76: #end-volume
77:
78: volume gfs-ds-io-cache
79: type performance/io-cache
80: option page-size 4KB
81: option cache-size 64MB
82: option priority *:0
83: option cache-timeout 1
84: # subvolumes gfs-ds-read-ahead
85: subvolumes gfs-ds-write-behind
86: end-volume
+------------------------------------------------------------------------------+
2009-03-30 18:25:31 E [xlator.c:201:_volume_option_value_validate] gfs-ds-io-cache: '4096' in 'option page-size 4KB' is out of range [16384 - 4194304]
2009-03-30 18:25:31 N [glusterfsd.c:1134:main] glusterfs: Successfully started
2009-03-30 18:25:31 N [client-protocol.c:6169:client_setvolume_cbk] gfs-ds-vu-hosting-tisiphone: connection and handshake succeeded
2009-03-30 18:25:31 N [client-protocol.c:6169:client_setvolume_cbk] gfs-ds-vu-hosting-tisiphone: connection and handshake succeeded
2009-03-30 18:25:31 N [client-protocol.c:6169:client_setvolume_cbk] gfs-ds-vu-hosting-hermes: connection and handshake succeeded
2009-03-30 18:25:31 N [client-protocol.c:6169:client_setvolume_cbk] gfs-ds-vu-hosting-hermes: connection and handshake succeeded
2009-03-30 18:25:37 N [client-protocol.c:6853:notify] gfs-ds-vu-hosting-tisiphone: disconnected
2009-03-30 18:25:42 N [client-protocol.c:6169:client_setvolume_cbk] gfs-ds-vu-hosting-tisiphone: connection and handshake succeeded
2009-03-30 18:25:42 N [client-protocol.c:6169:client_setvolume_cbk] gfs-ds-vu-hosting-tisiphone: connection and handshake succeeded
2009-03-30 18:27:52 E [client-protocol.c:537:client_ping_timer_expired] gfs-ds-vu-hosting-tisiphone: ping timer expired! bailing transport
2009-03-30 18:27:52 E [saved-frames.c:169:saved_frames_unwind] gfs-ds-vu-hosting-tisiphone: forced unwinding frame type(1) op(WRITE)
2009-03-30 18:27:52 E [saved-frames.c:169:saved_frames_unwind] gfs-ds-vu-hosting-tisiphone: forced unwinding frame type(2) op((null))
2009-03-30 18:27:52 E [client-protocol.c:635:client_ping_cbk] gfs-ds-vu-hosting-tisiphone: timer must have expired
2009-03-30 18:27:52 N [client-protocol.c:6853:notify] gfs-ds-vu-hosting-tisiphone: disconnected
2009-03-30 18:27:52 N [client-protocol.c:6169:client_setvolume_cbk] gfs-ds-vu-hosting-tisiphone: connection and handshake succeeded
2009-03-30 18:27:52 N [client-protocol.c:6169:client_setvolume_cbk] gfs-ds-vu-hosting-tisiphone: connection and handshake succeeded
----
tisiphone (server):
----
================================================================================
Version : glusterfs 2.0.0git built on Mar 30 2009 10:06:55
TLA Revision : git://git.sv.gnu.org/gluster.git
Starting Time: 2009-03-30 18:25:41
Command line : /usr/sbin/glusterfsd -N -f /etc/glusterfs/glusterfs-server.vu-hosting.vol -l /var/log/glusterfs/glusterfs-server.vu-hosting.log -L NORMAL
PID : 5899
System name : Linux
Nodename : tisiphone
Kernel Release : 2.6.29
Hardware Identifier: i686
Given volfile:
+------------------------------------------------------------------------------+
1: #######################################################
2: ### GlusterFS Server Volume Specification ###
3: #######################################################
4:
5: volume gfs-ds-posix
6: type storage/posix
7: option directory /mnt/glusterfs/vu-hosting
8: # option o-direct off
9: # option export-statfs-size yes
10: # option mandate-attribute yes
11: # option span-devices 1
12: end-volume
13:
14: volume gfs-ds-locks
15: type features/locks
16: option mandatory-locks off
17: subvolumes gfs-ds-posix
18: end-volume
19:
20: volume gfs-ds-io-threads
21: type performance/io-threads
22: option thread-count 8
23: subvolumes gfs-ds-locks
24: end-volume
25:
26: volume gfs-ds-server
27: type protocol/server
28: option transport-type tcp
29: option transport.socket.listen-port 6997
30: option auth.addr.gfs-ds-locks.allow 192.168.0.*,127.0.0.1
31: option auth.addr.gfs-ds-io-threads.allow 192.168.0.*,127.0.0.1
32: subvolumes gfs-ds-io-threads
33: end-volume
+------------------------------------------------------------------------------+
2009-03-30 18:25:41 N [glusterfsd.c:1134:main] glusterfs: Successfully started
2009-03-30 18:25:41 N [server-protocol.c:7513:mop_setvolume] gfs-ds-server: accepted client from 192.168.0.75:1013
2009-03-30 18:25:41 N [server-protocol.c:7513:mop_setvolume] gfs-ds-server: accepted client from 192.168.0.75:1012
2009-03-30 18:25:46 N [server-protocol.c:7513:mop_setvolume] gfs-ds-server: accepted client from 192.168.0.70:1017
2009-03-30 18:25:46 N [server-protocol.c:7513:mop_setvolume] gfs-ds-server: accepted client from 192.168.0.70:1016
2009-03-30 18:27:56 N [server-protocol.c:8268:notify] gfs-ds-server: 192.168.0.70:1016 disconnected
2009-03-30 18:27:56 N [server-protocol.c:7513:mop_setvolume] gfs-ds-server: accepted client from 192.168.0.70:1023
----
tisiphone (client):
----
================================================================================
Version : glusterfs 2.0.0git built on Mar 30 2009 10:06:55
TLA Revision : git://git.sv.gnu.org/gluster.git
Starting Time: 2009-03-30 18:25:41
Command line : /usr/sbin/glusterfsd -N -f /etc/glusterfs/glusterfs-client.vu-hosting.vol -l /var/log/glusterfs/glusterfs-client.vu-hosting.log -L NORMAL /var/www/vu-hosting
PID : 5934
System name : Linux
Nodename : tisiphone
Kernel Release : 2.6.29
Hardware Identifier: i686
Given volfile:
+------------------------------------------------------------------------------+
1: #######################################################
2: ### GlusterFS Client Volume Specification ###
3: #######################################################
4:
5: #volume gfs-ds-fuse
6: # type mount/fuse
7: # option direct-io-mode on
8: # option macfuse-local off
9: # option mount-point /var/www/vu-hosting
10: # option attribute-timeout 0.0
11: # option entry-timeout 0.0
12: #end-volume
13:
14: volume gfs-ds-vu-hosting-hermes
15: type protocol/client
16: option transport-type tcp
17: # option username
18: # option password
19: option remote-host 192.168.0.70
20: option remote-port 6997
21: option remote-subvolume gfs-ds-io-threads
22: option transport-timeout 10
23: option ping-timeout 5
24: end-volume
25:
26: volume gfs-ds-vu-hosting-tisiphone
27: type protocol/client
28: option transport-type tcp
29: # option username
30: # option password
31: option remote-host 192.168.0.75
32: option remote-port 6997
33: option remote-subvolume gfs-ds-io-threads
34: option transport-timeout 10
35: option ping-timeout 5
36: end-volume
37:
38: volume gfs-ds-replicate
39: type cluster/replicate
40: option data-self-heal on
41: option metadata-self-heal on
42: option entry-self-heal on
43: # option read-subvolume
44: # option favorite-child
45: option data-change-log on
46: option metadata-change-log on
47: option entry-change-log on
48: option data-lock-server-count 1
49: option metadata-lock-server-count 1
50: option entry-lock-server-count 1
51: subvolumes gfs-ds-vu-hosting-hermes gfs-ds-vu-hosting-tisiphone
52: end-volume
53:
54: volume gfs-ds-io-threads
55: type performance/io-threads
56: option thread-count 8
57: subvolumes gfs-ds-replicate
58: end-volume
59:
60: volume gfs-ds-write-behind
61: type performance/write-behind
62: option block-size 1MB
63: option cache-size 4MB
64: option flush-behind on
65: # opiton disable-for-first-nbytes 1
66: # option enable-O_SYNC false
67: subvolumes gfs-ds-io-threads
68: end-volume
69:
70: #volume gfs-ds-read-ahead
71: # type performance/read-ahead
72: # option page-size 65KB
73: # option page-count 16
74: # option force-atime-update off
75: # subvolumes gfs-ds-write-behind
76: #end-volume
77:
78: volume gfs-ds-io-cache
79: type performance/io-cache
80: option page-size 4KB
81: option cache-size 64MB
82: option priority *:0
83: option cache-timeout 1
84: # subvolumes gfs-ds-read-ahead
85: subvolumes gfs-ds-write-behind
86: end-volume
+------------------------------------------------------------------------------+
2009-03-30 18:25:41 E [xlator.c:201:_volume_option_value_validate] gfs-ds-io-cache: '4096' in 'option page-size 4KB' is out of range [16384 - 4194304]
2009-03-30 18:25:41 N [glusterfsd.c:1134:main] glusterfs: Successfully started
2009-03-30 18:25:41 N [client-protocol.c:6169:client_setvolume_cbk] gfs-ds-vu-hosting-hermes: connection and handshake succeeded
2009-03-30 18:25:41 N [client-protocol.c:6169:client_setvolume_cbk] gfs-ds-vu-hosting-hermes: connection and handshake succeeded
2009-03-30 18:25:41 N [client-protocol.c:6169:client_setvolume_cbk] gfs-ds-vu-hosting-tisiphone: connection and handshake succeeded
2009-03-30 18:25:41 N [client-protocol.c:6169:client_setvolume_cbk] gfs-ds-vu-hosting-tisiphone: connection and handshake succeeded
2009-03-30 18:29:03 E [client-protocol.c:537:client_ping_timer_expired] gfs-ds-vu-hosting-hermes: ping timer expired! bailing transport
2009-03-30 18:29:03 E [saved-frames.c:169:saved_frames_unwind] gfs-ds-vu-hosting-hermes: forced unwinding frame type(1) op(READ)
2009-03-30 18:29:03 E [saved-frames.c:169:saved_frames_unwind] gfs-ds-vu-hosting-hermes: forced unwinding frame type(2) op((null))
2009-03-30 18:29:03 E [client-protocol.c:635:client_ping_cbk] gfs-ds-vu-hosting-hermes: timer must have expired
2009-03-30 18:29:03 N [client-protocol.c:6853:notify] gfs-ds-vu-hosting-hermes: disconnected
2009-03-30 18:29:03 N [client-protocol.c:6169:client_setvolume_cbk] gfs-ds-vu-hosting-hermes: connection and handshake succeeded
2009-03-30 18:29:03 N [client-protocol.c:6169:client_setvolume_cbk] gfs-ds-vu-hosting-hermes: connection and handshake succeeded
----
I see the error with the 4KB being to low. I will fix that, but I don't think that this is the problem. Or could it be?
I don't know why the client has a ping timeout? The system is connected with a brand new HP ProCurve 24 Port Gigabit Switch.
The network cards use 7200 MTU.
> Vikas
>
Steve
> --
> Engineer - Z Research
> http://gluster.com/
--
Neu: GMX FreeDSL Komplettanschluss mit DSL 6.000 Flatrate + Telefonanschluss f?r nur 17,95 Euro/mtl.!* http://dsl.gmx.de/?ac=OM.AD.PD003K11308T4569a
