Greetings List,
I've searched through the Gluster wiki and a lot of threads to try to answer this question, but so far no real luck.
Simply put - is it better to have replication handled by the clients, or by the bricks themselves?
Volgen for a raid 1 solution creates a config file that does the mirroring on the client side - which I would take as an implicit endorsement from the Gluster team (great team, BTW). However, it seems to me that if the bricks replicated between themselves on our 10Gb storage network, it could save a lot of bandwidth for the clients and conceivably save them CPU cycles an I/O as well.
Client machines have 1Gb connections to the storage network, and are running CentOS 5.2.
Server machines have 10Gb connections to the storage network, and are running CentOS 5.4.
Glusterfs.vol:
## file auto generated by /usr/bin/glusterfs-volgen (mount.vol)
# Cmd line:
# $ /usr/bin/glusterfs-volgen --name testfs --raid 1 jc1letgfs13-pfs1:/export/read-write jc1letgfs14-pfs1:/export/read-write jc1letgfs15-pfs1:/export/read-write jc1letgfs16-pfs1:/export/read-write jc1letgfs17-pfs1:/export/read-write jc1letgfs18-pfs1:/export/read-write
# RAID 1
# TRANSPORT-TYPE tcp
volume jc1letgfs17-pfs1-1
type protocol/client
option transport-type tcp
option remote-host jc1letgfs17-pfs1
option transport.socket.nodelay on
option transport.remote-port 6996
option remote-subvolume brick1
end-volume
volume jc1letgfs18-pfs1-1
type protocol/client
option transport-type tcp
option remote-host jc1letgfs18-pfs1
option transport.socket.nodelay on
option transport.remote-port 6996
option remote-subvolume brick1
end-volume
volume jc1letgfs13-pfs1-1
type protocol/client
option transport-type tcp
option remote-host jc1letgfs13-pfs1
option transport.socket.nodelay on
option transport.remote-port 6996
option remote-subvolume brick1
end-volume
volume jc1letgfs15-pfs1-1
type protocol/client
option transport-type tcp
option remote-host jc1letgfs15-pfs1
option transport.socket.nodelay on
option transport.remote-port 6996
option remote-subvolume brick1
end-volume
volume jc1letgfs16-pfs1-1
type protocol/client
option transport-type tcp
option remote-host jc1letgfs16-pfs1
option transport.socket.nodelay on
option transport.remote-port 6996
option remote-subvolume brick1
end-volume
volume jc1letgfs14-pfs1-1
type protocol/client
option transport-type tcp
option remote-host jc1letgfs14-pfs1
option transport.socket.nodelay on
option transport.remote-port 6996
option remote-subvolume brick1
end-volume
volume mirror-0
type cluster/replicate
subvolumes jc1letgfs13-pfs1-1 jc1letgfs14-pfs1-1
end-volume
volume mirror-1
type cluster/replicate
subvolumes jc1letgfs15-pfs1-1 jc1letgfs16-pfs1-1
end-volume
volume mirror-2
type cluster/replicate
subvolumes jc1letgfs17-pfs1-1 jc1letgfs18-pfs1-1
end-volume
volume distribute
type cluster/distribute
subvolumes mirror-0 mirror-1 mirror-2
end-volume
volume readahead
type performance/read-ahead
option page-count 4
subvolumes distribute
end-volume
volume iocache
type performance/io-cache
option cache-size `echo $(( $(grep 'MemTotal' /proc/meminfo | sed 's/[^0-9]//g') / 5120 ))`MB
option cache-timeout 1
subvolumes readahead
end-volume
volume quickread
type performance/quick-read
option cache-timeout 1
option max-file-size 64kB
subvolumes iocache
end-volume
volume writebehind
type performance/write-behind
option cache-size 4MB
subvolumes quickread
end-volume
volume statprefetch
type performance/stat-prefetch
subvolumes writebehind
end-volume
Glusterfsd.vol:
## file auto generated by /usr/bin/glusterfs-volgen (export.vol)
# Cmd line:
# $ /usr/bin/glusterfs-volgen --name testfs jc1letgfs13-pfs1:/export/read-write jc1letgfs14-pfs1:/export/read-write jc1letgfs15-pfs1:/export/read-write
volume posix1
type storage/posix
option directory /export/read-write
end-volume
volume locks1
type features/locks
subvolumes posix1
end-volume
volume brick1
type performance/io-threads
option thread-count 8
subvolumes locks1
end-volume
volume server-tcp
type protocol/server
option transport-type tcp
option auth.addr.brick1.allow *
option transport.socket.listen-port 6996
option transport.socket.nodelay on
subvolumes brick1
end-volume
James Burnash
DISCLAIMER:
This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this in error, please immediately notify me and permanently delete the original and any copy of any e-mail and any printout thereof. E-mail transmission cannot be guaranteed to be secure or error-free. The sender therefore does not accept liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission.
NOTICE REGARDING PRIVACY AND CONFIDENTIALITY Knight Capital Group may, at its discretion, monitor and review the content of all e-mail communications. http://www.knight.com
