Hello,
I have 128 physically identical blades, with 1GbE uplink per blade,
and 10GbE between chassis ( 32 blades per chassis ). Each node will
have a 80GB gluster partition. Dual-quad core intel Xeons, 24GB RAM.
The goal is to use gluster as a cache for files used by render
applications. All files in gluster could be re-generated or retrieved
from the upstream file server.
My first volume config attempt is 64 replicated volumes with partner
pairs on different chassis.
Is replicating a performance hit? Do reads balance between replication nodes?
Would NUFA make more sense for this set-up?
Here is my config, any advice appreciated.
Thank you,
-Barry
>>>>
volume c001b17-1
type protocol/client
option transport-type tcp
option remote-host c001b17
option transport.socket.nodelay on
option transport.remote-port 6996
option remote-subvolume brick1
option ping-timeout 5
end-volume
.
<snip>
.
volume c004b48-1
type protocol/client
option transport-type tcp
option remote-host c004b48
option transport.socket.nodelay on
option transport.remote-port 6996
option remote-subvolume brick1
option ping-timeout 5
end-volume
volume replicate001-17
type cluster/replicate
subvolumes c001b17-1 c002b17-1
end-volume
.
<snip>
.
volume replicate001-48
type cluster/replicate
subvolumes c001b48-1 c002b48-1
end-volume
volume replicate003-17
type cluster/replicate
subvolumes c003b17-1 c004b17-1
end-volume
.
<snip>
.
volume replicate003-48
type cluster/replicate
subvolumes c003b48-1 c004b48-1
end-volume
volume distribute
type cluster/distribute
subvolumes replicate001-17 replicate001-18 replicate001-19
replicate001-20 replicate001-21 replicate001-22 replicate001-23
replicate001-24 replicate001-25 replicate001-26 replicate001-27
replicate001-28 replicate001-29 replicate001-30 replicate001-31
replicate001-32 replicate001-33 replicate001-34 replicate001-35
replicate001-36 replicate001-37 replicate001-38 replicate001-39
replicate001-40 replicate001-41 replicate001-42 replicate001-43
replicate001-44 replicate001-45 replicate001-46 replicate001-47
replicate001-48 replicate003-17 replicate003-18 replicate003-19
replicate003-20 replicate003-21 replicate003-22 replicate003-23
replicate003-24 replicate003-25 replicate003-26 replicate003-27
replicate003-28 replicate003-29 replicate003-30 replicate003-31
replicate003-32 replicate003-33 replicate003-34 replicate003-35
replicate003-36 replicate003-37 replicate003-38 replicate003-39
replicate003-40 replicate003-41 replicate003-42 replicate003-43
replicate003-44 replicate003-45 replicate003-46 replicate003-47
replicate003-48
end-volume
volume writebehind
type performance/write-behind
option cache-size 64MB
option flush-behind on
subvolumes distribute
end-volume
volume readahead
type performance/read-ahead
option page-count 4
subvolumes writebehind
end-volume
volume iocache
type performance/io-cache
option cache-size 128MB
option cache-timeout 10
subvolumes readahead
end-volume
volume quickread
type performance/quick-read
option cache-timeout 1
option max-file-size 64kB
subvolumes iocache
end-volume
volume statprefetch
type performance/stat-prefetch
subvolumes quickread
end-volume
