== Data Copying when a new brick is added. ==
Hi. I'm a first time glusterfs user, and I'm trying to simulate what will
happen when I need to add more bricks for storage capacity. My config files
are below but I'll try and explain what is going on. I have 2 machines with
2 hard drives in each. I created a replicated storage system where machine
a replicates to machine b. Everything worked fine when only 1 harddrive
was set up on each machine. I filled up the harddrives to the point where
there is only 1.5 gigs free on each one. They match because it was all
replicated.
After this, I added in the configuration for the second harddrive in each
machine. The machines are still replicating, so the harddrives should be
distributed, which is what I tried. All is well with the new configuration
if I add new files to the system. It correctly sees that the old harddrives
are full and puts all the new files onto the second empty drive.
However, if I go into the mount point on the client machine and do a
directory listing (or the shell autocomplete attempts to read it) something
strange happens. Machine 1 is fine, but machine 2 starts to copy over files
from the full harddrive to the empty one. It's still copying this time
after I retried the configuration, but the first time it copied about half
the data over to the second harddrive. The first harddrive remained full
though.
Has anyone experienced this, or can see something wrong with my
configuration files? The server files are exactly the same on both machines
and the machines have the same hardware components and OS (Ubuntu) as well.
Any help would be appreciated.
===========================================================
Server Configuration
===========================================================
volume posix1
type storage/posix
option directory /data/export
end-volume
volume posix2
type storage/posix
option directory /media/disk2
end-volume
volume locks
type features/locks
subvolumes posix1
end-volume
volume locks2
type features/locks
subvolumes posix2
end-volume
volume brick
type performance/io-threads
option thread-count 8
subvolumes locks
end-volume
volume brick2
type performance/io-threads
option thread-count 8
subvolumes locks2
end-volume
volume server
type protocol/server
option transport-type tcp
option auth.addr.brick.allow 192.168.0.191,192.168.0.192
option auth.addr.brick2.allow 192.168.0.191,192.168.0.192
subvolumes brick brick2
end-volume
===========================================================
Client Configuration
===========================================================
volume remote1a
type protocol/client
option transport-type tcp
option remote-host 192.168.0.151
option remote-subvolume brick
end-volume
volume remote1b
type protocol/client
option transport-type tcp
option remote-host 192.168.0.151
option remote-subvolume brick2
end-volume
volume remote2a
type protocol/client
option transport-type tcp
option remote-host 192.168.0.152
option remote-subvolume brick
end-volume
volume remote2b
type protocol/client
option transport-type tcp
option remote-host 192.168.0.152
option remote-subvolume brick2
end-volume
volume unify1
type cluster/distribute
subvolumes remote1a remote1b
option scheduler alu # use the ALU scheduler
option alu.limits.min-free-disk 5% # Don't create files one a
volume with less than 5% free diskspace
option alu.limits.max-open-files 10000 # Don't create files on a
volume with more than 10000 files open
option alu.order
disk-usage:read-usage:write-usage:open-files-usage:disk-speed-usage
option alu.disk-usage.entry-threshold 3GB # Kick in if the discrepancy
in disk-usage between volumes is more than 3GB
option alu.disk-usage.exit-threshold 60MB # Don't stop writing to the
least-used volume until the discrepancy is 1988MB
option alu.open-files-usage.entry-threshold 1024 # Kick in if the
discrepancy in open files is 1024
option alu.open-files-usage.exit-threshold 32 # Don't stop until 992
files have been written the least-used volume
option alu.stat-refresh.interval 60sec # Refresh the statistics used
for decision-making every 60 seconds
end-volume
volume unify2
type cluster/distribute
subvolumes remote2a remote2b
option scheduler alu # use the ALU scheduler
option alu.limits.min-free-disk 5% # Don't create files one a
volume with less than 5% free diskspace
option alu.limits.max-open-files 10000 # Don't create files on a
volume with more than 10000 files open
option alu.order
disk-usage:read-usage:write-usage:open-files-usage:disk-speed-usage
option alu.disk-usage.entry-threshold 3GB # Kick in if the discrepancy
in disk-usage between volumes is more than 3GB
option alu.disk-usage.exit-threshold 60MB # Don't stop writing to the
least-used volume until the discrepancy is 1988MB
option alu.open-files-usage.entry-threshold 1024 # Kick in if the
discrepancy in open files is 1024
option alu.open-files-usage.exit-threshold 32 # Don't stop until 992
files have been written the least-used volume
option alu.stat-refresh.interval 60sec # Refresh the statistics used
for decision-making every 60 seconds
end-volume
volume replicate
type cluster/afr
subvolumes unify1 unify2
end-volume
volume writebehind
type performance/write-behind
option window-size 1MB
subvolumes replicate
end-volume
volume cache
type performance/io-cache
option cache-size 512MB
subvolumes writebehind
end-volume
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gluster.org/pipermail/gluster-users/attachments/20110514/3d4f1b75/attachment.htm>
