FYI - this is fixed, samppah on IRC suggesting updating to the latest (3.04) and that fixed it, and an issue with not being able to delete files. P On Tue, May 4, 2010 at 10:26 AM, phil cryer <phil at cryer.us> wrote: > I have a distributed/replicated setup with Glusterfs 3.0.2, that I'm > testing on 4 servers, each with access to /mnt/gluster (which consists > of all directories /mnt/data01 - data24) on each server. ?I'm using > configs I built from volgen, but every time I access a file (via an > 'ls -l') for the first time, I get all of these messages in my logs on > each server: > > [2010-05-04 10:50:30] W [posix.c:246:posix_lstat_with_gen] posix1: > Access to /mnt/data01//.. (on dev 16771) is crossing device (2257) > [2010-05-04 10:50:30] W [posix.c:246:posix_lstat_with_gen] posix3: > Access to /mnt/data03//.. (on dev 16771) is crossing device (2241) > [2010-05-04 10:50:30] W [posix.c:246:posix_lstat_with_gen] posix5: > Access to /mnt/data05//.. (on dev 16771) is crossing device (2225) > [2010-05-04 10:50:30] W [posix.c:246:posix_lstat_with_gen] posix5: > Access to /mnt/data05//.. (on dev 16771) is crossing device (2225) > [2010-05-04 10:50:30] W [posix.c:246:posix_lstat_with_gen] posix7: > Access to /mnt/data07//.. (on dev 16771) is crossing device (16705) > [2010-05-04 10:50:30] W [posix.c:246:posix_lstat_with_gen] posix7: > Access to /mnt/data07//.. (on dev 16771) is crossing device (16705) > [2010-05-04 10:50:30] W [posix.c:246:posix_lstat_with_gen] posix9: > Access to /mnt/data09//.. (on dev 16771) is crossing device (2161) > [2010-05-04 10:50:30] W [posix.c:246:posix_lstat_with_gen] posix11: > Access to /mnt/data11//.. (on dev 16771) is crossing device (2113) > [2010-05-04 10:50:30] W [posix.c:246:posix_lstat_with_gen] posix11: > Access to /mnt/data11//.. (on dev 16771) is crossing device (2113) > [2010-05-04 10:50:30] W [posix.c:246:posix_lstat_with_gen] posix13: > Access to /mnt/data13//.. (on dev 16771) is crossing device (2081) > [2010-05-04 10:50:30] W [posix.c:246:posix_lstat_with_gen] posix15: > Access to /mnt/data15//.. (on dev 16771) is crossing device (16689) > [2010-05-04 10:50:30] W [posix.c:246:posix_lstat_with_gen] posix15: > Access to /mnt/data15//.. (on dev 16771) is crossing device (16689) > [2010-05-04 10:50:30] W [posix.c:246:posix_lstat_with_gen] posix17: > Access to /mnt/data17//.. (on dev 16771) is crossing device (2289) > [2010-05-04 10:50:30] W [posix.c:246:posix_lstat_with_gen] posix19: > Access to /mnt/data19//.. (on dev 16771) is crossing device (2049) > [2010-05-04 10:50:30] W [posix.c:246:posix_lstat_with_gen] posix21: > Access to /mnt/data21//.. (on dev 16771) is crossing device (16737) > [2010-05-04 10:50:30] W [posix.c:246:posix_lstat_with_gen] posix23: > Access to /mnt/data23//.. (on dev 16771) is crossing device (2065) > [2010-05-04 10:50:30] W [posix.c:246:posix_lstat_with_gen] posix23: > Access to /mnt/data23//.. (on dev 16771) is crossing device (2065) > > What do I have setup incorrectly here? ?My config files are below, thanks. > > > # cat /etc/glusterfs/glusterfs.vol > > ######################################## > ## glusterfs.vol > ## phil.cryer at mobot.org > ######################################## > # ?128.128.164.74 ?clustr-01 > # ?128.128.164.89 ?clustr-02 > # ?128.128.164.215 clustr-03 > # ?128.128.164.217 clustr-04 > # ?128.128.164.218 clustr-05 > # ?128.128.164.219 clustr-06 > ######################################## > # RAID 1 > # TRANSPORT-TYPE tcp > ######################################## > ######################################## > volume clustr-03-1 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-03 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick1 > end-volume > volume clustr-03-2 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-03 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick2 > end-volume > volume clustr-03-3 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-03 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick3 > end-volume > volume clustr-03-4 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-03 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick4 > end-volume > volume clustr-03-5 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-03 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick5 > end-volume > volume clustr-03-6 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-03 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick6 > end-volume > volume clustr-03-7 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-03 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick7 > end-volume > volume clustr-03-8 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-03 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick8 > end-volume > volume clustr-03-9 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-03 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick9 > end-volume > volume clustr-03-10 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-03 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick10 > end-volume > volume clustr-03-11 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-03 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick11 > end-volume > volume clustr-03-12 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-03 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick12 > end-volume > volume clustr-03-13 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-03 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick13 > end-volume > volume clustr-03-14 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-03 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick14 > end-volume > volume clustr-03-15 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-03 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick15 > end-volume > volume clustr-03-16 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-03 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick16 > end-volume > volume clustr-03-17 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-03 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick17 > end-volume > volume clustr-03-18 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-03 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick18 > end-volume > volume clustr-03-19 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-03 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick19 > end-volume > volume clustr-03-20 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-03 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick20 > end-volume > volume clustr-03-21 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-03 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick21 > end-volume > volume clustr-03-22 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-03 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick22 > end-volume > volume clustr-03-23 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-03 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick23 > end-volume > volume clustr-03-24 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-03 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick24 > end-volume > ######################################## > volume clustr-06-1 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-06 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick1 > end-volume > volume clustr-06-2 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-06 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick2 > end-volume > volume clustr-06-3 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-06 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick3 > end-volume > volume clustr-06-4 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-06 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick4 > end-volume > volume clustr-06-5 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-06 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick5 > end-volume > volume clustr-06-6 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-06 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick6 > end-volume > volume clustr-06-7 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-06 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick7 > end-volume > volume clustr-06-8 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-06 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick8 > end-volume > volume clustr-06-9 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-06 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick9 > end-volume > volume clustr-06-10 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-06 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick10 > end-volume > volume clustr-06-11 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-06 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick11 > end-volume > volume clustr-06-12 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-06 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick12 > end-volume > volume clustr-06-13 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-06 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick13 > end-volume > volume clustr-06-14 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-06 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick14 > end-volume > volume clustr-06-15 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-06 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick15 > end-volume > volume clustr-06-16 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-06 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick16 > end-volume > volume clustr-06-17 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-06 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick17 > end-volume > volume clustr-06-18 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-06 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick18 > end-volume > volume clustr-06-19 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-06 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick19 > end-volume > volume clustr-06-20 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-06 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick20 > end-volume > volume clustr-06-21 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-06 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick21 > end-volume > volume clustr-06-22 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-06 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick22 > end-volume > volume clustr-06-23 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-06 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick23 > end-volume > volume clustr-06-24 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-06 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick24 > end-volume > ######################################## > volume clustr-01-1 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-01 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick1 > end-volume > volume clustr-01-2 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-01 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick2 > end-volume > volume clustr-01-3 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-01 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick3 > end-volume > volume clustr-01-4 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-01 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick4 > end-volume > volume clustr-01-5 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-01 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick5 > end-volume > volume clustr-01-6 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-01 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick6 > end-volume > volume clustr-01-7 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-01 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick7 > end-volume > volume clustr-01-8 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-01 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick8 > end-volume > volume clustr-01-9 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-01 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick9 > end-volume > volume clustr-01-10 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-01 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick10 > end-volume > volume clustr-01-11 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-01 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick11 > end-volume > volume clustr-01-12 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-01 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick12 > end-volume > volume clustr-01-13 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-01 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick13 > end-volume > volume clustr-01-14 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-01 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick14 > end-volume > volume clustr-01-15 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-01 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick15 > end-volume > volume clustr-01-16 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-01 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick16 > end-volume > volume clustr-01-17 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-01 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick17 > end-volume > volume clustr-01-18 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-01 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick18 > end-volume > volume clustr-01-19 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-01 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick19 > end-volume > volume clustr-01-20 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-01 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick20 > end-volume > volume clustr-01-21 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-01 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick21 > end-volume > volume clustr-01-22 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-01 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick22 > end-volume > volume clustr-01-23 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-01 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick23 > end-volume > volume clustr-01-24 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-01 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick24 > end-volume > ######################################## > volume clustr-02-1 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-02 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick1 > end-volume > volume clustr-02-2 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-02 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick2 > end-volume > volume clustr-02-3 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-02 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick3 > end-volume > volume clustr-02-4 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-02 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick4 > end-volume > volume clustr-02-5 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-02 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick5 > end-volume > volume clustr-02-6 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-02 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick6 > end-volume > volume clustr-02-7 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-02 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick7 > end-volume > volume clustr-02-8 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-02 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick8 > end-volume > volume clustr-02-9 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-02 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick9 > end-volume > volume clustr-02-10 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-02 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick10 > end-volume > volume clustr-02-11 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-02 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick11 > end-volume > volume clustr-02-12 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-02 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick12 > end-volume > volume clustr-02-13 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-02 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick13 > end-volume > volume clustr-02-14 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-02 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick14 > end-volume > volume clustr-02-15 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-02 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick15 > end-volume > volume clustr-02-16 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-02 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick16 > end-volume > volume clustr-02-17 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-02 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick17 > end-volume > volume clustr-02-18 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-02 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick18 > end-volume > volume clustr-02-19 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-02 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick19 > end-volume > volume clustr-02-20 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-02 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick20 > end-volume > volume clustr-02-21 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-02 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick21 > end-volume > volume clustr-02-22 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-02 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick22 > end-volume > volume clustr-02-23 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-02 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick23 > end-volume > volume clustr-02-24 > ? ?type protocol/client > ? ?option transport-type tcp > ? ?option remote-host clustr-02 > ? ?option transport.socket.nodelay on > ? ?option transport.remote-port 6996 > ? ?option remote-subvolume brick24 > end-volume > ######################################## > ######################################## > volume mirror-0 > ? ?type cluster/replicate > ? ?subvolumes clustr-03-1 clustr-03-2 > end-volume > volume mirror-1 > ? ?type cluster/replicate > ? ?subvolumes clustr-03-3 clustr-03-4 > end-volume > volume mirror-2 > ? ?type cluster/replicate > ? ?subvolumes clustr-03-5 clustr-03-6 > end-volume > volume mirror-3 > ? ?type cluster/replicate > ? ?subvolumes clustr-03-7 clustr-03-8 > end-volume > volume mirror-4 > ? ?type cluster/replicate > ? ?subvolumes clustr-03-9 clustr-03-10 > end-volume > volume mirror-5 > ? ?type cluster/replicate > ? ?subvolumes clustr-03-11 clustr-03-12 > end-volume > volume mirror-6 > ? ?type cluster/replicate > ? ?subvolumes clustr-03-13 clustr-03-14 > end-volume > volume mirror-7 > ? ?type cluster/replicate > ? ?subvolumes clustr-03-15 clustr-03-16 > end-volume > volume mirror-8 > ? ?type cluster/replicate > ? ?subvolumes clustr-03-17 clustr-03-18 > end-volume > volume mirror-9 > ? ?type cluster/replicate > ? ?subvolumes clustr-03-19 clustr-03-20 > end-volume > volume mirror-10 > ? ?type cluster/replicate > ? ?subvolumes clustr-03-21 clustr-03-22 > end-volume > volume mirror-11 > ? ?type cluster/replicate > ? ?subvolumes clustr-03-23 clustr-03-24 > end-volume > ######################################## > volume mirror-12 > ? ?type cluster/replicate > ? ?subvolumes clustr-06-1 clustr-06-2 > end-volume > volume mirror-13 > ? ?type cluster/replicate > ? ?subvolumes clustr-06-3 clustr-06-4 > end-volume > volume mirror-14 > ? ?type cluster/replicate > ? ?subvolumes clustr-06-5 clustr-06-6 > end-volume > volume mirror-15 > ? ?type cluster/replicate > ? ?subvolumes clustr-06-7 clustr-06-8 > end-volume > volume mirror-16 > ? ?type cluster/replicate > ? ?subvolumes clustr-06-9 clustr-06-10 > end-volume > volume mirror-17 > ? ?type cluster/replicate > ? ?subvolumes clustr-06-11 clustr-06-12 > end-volume > volume mirror-18 > ? ?type cluster/replicate > ? ?subvolumes clustr-06-13 clustr-06-14 > end-volume > volume mirror-19 > ? ?type cluster/replicate > ? ?subvolumes clustr-06-15 clustr-06-16 > end-volume > volume mirror-20 > ? ?type cluster/replicate > ? ?subvolumes clustr-06-17 clustr-06-18 > end-volume > volume mirror-21 > ? ?type cluster/replicate > ? ?subvolumes clustr-06-19 clustr-06-20 > end-volume > volume mirror-22 > ? ?type cluster/replicate > ? ?subvolumes clustr-06-21 clustr-06-22 > end-volume > volume mirror-23 > ? ?type cluster/replicate > ? ?subvolumes clustr-06-23 clustr-06-24 > end-volume > ######################################## > volume mirror-24 > ? ?type cluster/replicate > ? ?subvolumes clustr-01-1 clustr-01-2 > end-volume > volume mirror-25 > ? ?type cluster/replicate > ? ?subvolumes clustr-01-3 clustr-01-4 > end-volume > volume mirror-26 > ? ?type cluster/replicate > ? ?subvolumes clustr-01-5 clustr-01-6 > end-volume > volume mirror-27 > ? ?type cluster/replicate > ? ?subvolumes clustr-01-7 clustr-01-8 > end-volume > volume mirror-28 > ? ?type cluster/replicate > ? ?subvolumes clustr-01-9 clustr-01-10 > end-volume > volume mirror-29 > ? ?type cluster/replicate > ? ?subvolumes clustr-01-11 clustr-01-12 > end-volume > volume mirror-30 > ? ?type cluster/replicate > ? ?subvolumes clustr-01-13 clustr-01-14 > end-volume > volume mirror-31 > ? ?type cluster/replicate > ? ?subvolumes clustr-01-15 clustr-01-16 > end-volume > volume mirror-32 > ? ?type cluster/replicate > ? ?subvolumes clustr-01-17 clustr-01-18 > end-volume > volume mirror-33 > ? ?type cluster/replicate > ? ?subvolumes clustr-01-19 clustr-01-20 > end-volume > volume mirror-34 > ? ?type cluster/replicate > ? ?subvolumes clustr-01-21 clustr-01-22 > end-volume > volume mirror-35 > ? ?type cluster/replicate > ? ?subvolumes clustr-01-23 clustr-01-24 > end-volume > ######################################## > volume mirror-36 > ? ?type cluster/replicate > ? ?subvolumes clustr-02-1 clustr-02-2 > end-volume > volume mirror-37 > ? ?type cluster/replicate > ? ?subvolumes clustr-02-3 clustr-02-4 > end-volume > volume mirror-38 > ? ?type cluster/replicate > ? ?subvolumes clustr-02-5 clustr-02-6 > end-volume > volume mirror-39 > ? ?type cluster/replicate > ? ?subvolumes clustr-02-7 clustr-02-8 > end-volume > volume mirror-40 > ? ?type cluster/replicate > ? ?subvolumes clustr-02-9 clustr-02-10 > end-volume > volume mirror-41 > ? ?type cluster/replicate > ? ?subvolumes clustr-02-11 clustr-02-12 > end-volume > volume mirror-42 > ? ?type cluster/replicate > ? ?subvolumes clustr-02-13 clustr-02-14 > end-volume > volume mirror-43 > ? ?type cluster/replicate > ? ?subvolumes clustr-02-15 clustr-02-16 > end-volume > volume mirror-44 > ? ?type cluster/replicate > ? ?subvolumes clustr-02-17 clustr-02-18 > end-volume > volume mirror-45 > ? ?type cluster/replicate > ? ?subvolumes clustr-02-19 clustr-02-20 > end-volume > volume mirror-46 > ? ?type cluster/replicate > ? ?subvolumes clustr-02-21 clustr-02-22 > end-volume > volume mirror-47 > ? ?type cluster/replicate > ? ?subvolumes clustr-02-23 clustr-02-24 > end-volume > ######################################## > ######################################## > volume distribute > ? ?type cluster/distribute > ? ?subvolumes mirror-0 mirror-1 mirror-2 mirror-3 mirror-4 mirror-5 > mirror-6 mirror-7 mirror-8 mirror-9 mirror-10 mirror-11 mirror-12 > mirror-13 mirror-14 mirror-15 mirror-16 mirror-17 mirror-18 mirror-19 > mirror-20 mirror-21 mirror-22 mirror-23 mirror-24 mirror-25 mirror-26 > mirror-27 mirror-28 mirror-29 mirror-30 mirror-31 mirror-32 mirror-33 > mirror-34 mirror-35 mirror-36 mirror-37 mirror-38 mirror-39 mirror-40 > mirror-41 mirror-42 mirror-43 mirror-44 mirror-45 mirror-46 mirror-47 > end-volume > ######################################## > ######################################## > volume writebehind > ? ?type performance/write-behind > ? ?option cache-size 4MB > ? ?subvolumes distribute > end-volume > volume readahead > ? ?type performance/read-ahead > ? ?option page-count 4 > ? ?subvolumes writebehind > end-volume > volume iocache > ? ?type performance/io-cache > ? ?option cache-size `grep 'MemTotal' /proc/meminfo ?| awk '{print $2 > * 0.2 / 1024}' | cut -f1 -d.`MB > ? ?option cache-timeout 1 > ? ?subvolumes readahead > end-volume > volume quickread > ? ?type performance/quick-read > ? ?option cache-timeout 1 > ? ?option max-file-size 64kB > ? ?subvolumes iocache > end-volume > volume statprefetch > ? ?type performance/stat-prefetch > ? ?subvolumes quickread > end-volume > ######################################## > ######################################## > > > > > > # cat /etc/glusterfs/glusterfsd.vol > > ## file auto generated by /usr/local/bin/glusterfs-volgen (export.vol) > # Cmd line: > # $ /usr/local/bin/glusterfs-volgen --name store-0405 --raid 1 > clustr-04:/mnt/data01 clustr-04:/mnt/data02 clustr-04:/mnt/data03 > clustr-04:/mnt/data04 clustr-04:/mnt/data05 clustr-04:/mnt/data06 > clustr-04:/mnt/data07 clustr-04:/mnt/data08 clustr-04:/mnt/data09 > clustr-04:/mnt/data10 clustr-04:/mnt/data11 clustr-04:/mnt/data12 > clustr-04:/mnt/data13 clustr-04:/mnt/data14 clustr-04:/mnt/data15 > clustr-04:/mnt/data16 clustr-04:/mnt/data17 clustr-04:/mnt/data18 > clustr-04:/mnt/data19 clustr-04:/mnt/data20 clustr-04:/mnt/data21 > clustr-04:/mnt/data22 clustr-04:/mnt/data23 clustr-04:/mnt/data24 > clustr-05:/mnt/data01 clustr-05:/mnt/data02 clustr-05:/mnt/data03 > clustr-05:/mnt/data04 clustr-05:/mnt/data05 clustr-05:/mnt/data06 > clustr-05:/mnt/data07 clustr-05:/mnt/data08 clustr-05:/mnt/data09 > clustr-05:/mnt/data10 clustr-05:/mnt/data11 clustr-05:/mnt/data12 > clustr-05:/mnt/data13 clustr-05:/mnt/data14 clustr-05:/mnt/data15 > clustr-05:/mnt/data16 clustr-05:/mnt/data17 clustr-05:/mnt/data18 > clustr-05:/mnt/data19 clustr-05:/mnt/data20 clustr-05:/mnt/data21 > clustr-05:/mnt/data22 clustr-05:/mnt/data23 clustr-05:/mnt/data24 > ######################################## > ## glusterfs.vol > ## phil.cryer at mobot.org > ######################################## > # ?128.128.164.74 ?clustr-01 > # ?128.128.164.89 ?clustr-02 > # ?128.128.164.215 clustr-03 > # ?128.128.164.217 clustr-04 > # ?128.128.164.218 clustr-05 > # ?128.128.164.219 clustr-06 > ######################################## > volume posix1 > ?type storage/posix > ?option directory /mnt/data01 > end-volume > > volume locks1 > ? ?type features/locks > ? ?subvolumes posix1 > end-volume > > volume brick1 > ? ?type performance/io-threads > ? ?option thread-count 8 > ? ?subvolumes locks1 > end-volume > > volume posix2 > ?type storage/posix > ?option directory /mnt/data02 > end-volume > > volume locks2 > ? ?type features/locks > ? ?subvolumes posix2 > end-volume > > volume brick2 > ? ?type performance/io-threads > ? ?option thread-count 8 > ? ?subvolumes locks2 > end-volume > > volume posix3 > ?type storage/posix > ?option directory /mnt/data03 > end-volume > > volume locks3 > ? ?type features/locks > ? ?subvolumes posix3 > end-volume > > volume brick3 > ? ?type performance/io-threads > ? ?option thread-count 8 > ? ?subvolumes locks3 > end-volume > > volume posix4 > ?type storage/posix > ?option directory /mnt/data04 > end-volume > > volume locks4 > ? ?type features/locks > ? ?subvolumes posix4 > end-volume > > volume brick4 > ? ?type performance/io-threads > ? ?option thread-count 8 > ? ?subvolumes locks4 > end-volume > > volume posix5 > ?type storage/posix > ?option directory /mnt/data05 > end-volume > > volume locks5 > ? ?type features/locks > ? ?subvolumes posix5 > end-volume > > volume brick5 > ? ?type performance/io-threads > ? ?option thread-count 8 > ? ?subvolumes locks5 > end-volume > > volume posix6 > ?type storage/posix > ?option directory /mnt/data06 > end-volume > > volume locks6 > ? ?type features/locks > ? ?subvolumes posix6 > end-volume > > volume brick6 > ? ?type performance/io-threads > ? ?option thread-count 8 > ? ?subvolumes locks6 > end-volume > > volume posix7 > ?type storage/posix > ?option directory /mnt/data07 > end-volume > > volume locks7 > ? ?type features/locks > ? ?subvolumes posix7 > end-volume > > volume brick7 > ? ?type performance/io-threads > ? ?option thread-count 8 > ? ?subvolumes locks7 > end-volume > > volume posix8 > ?type storage/posix > ?option directory /mnt/data08 > end-volume > > volume locks8 > ? ?type features/locks > ? ?subvolumes posix8 > end-volume > > volume brick8 > ? ?type performance/io-threads > ? ?option thread-count 8 > ? ?subvolumes locks8 > end-volume > > volume posix9 > ?type storage/posix > ?option directory /mnt/data09 > end-volume > > volume locks9 > ? ?type features/locks > ? ?subvolumes posix9 > end-volume > > volume brick9 > ? ?type performance/io-threads > ? ?option thread-count 8 > ? ?subvolumes locks9 > end-volume > > volume posix10 > ?type storage/posix > ?option directory /mnt/data10 > end-volume > > volume locks10 > ? ?type features/locks > ? ?subvolumes posix10 > end-volume > > volume brick10 > ? ?type performance/io-threads > ? ?option thread-count 8 > ? ?subvolumes locks10 > end-volume > > volume posix11 > ?type storage/posix > ?option directory /mnt/data11 > end-volume > > volume locks11 > ? ?type features/locks > ? ?subvolumes posix11 > end-volume > > volume brick11 > ? ?type performance/io-threads > ? ?option thread-count 8 > ? ?subvolumes locks11 > end-volume > > volume posix12 > ?type storage/posix > ?option directory /mnt/data12 > end-volume > > volume locks12 > ? ?type features/locks > ? ?subvolumes posix12 > end-volume > > volume brick12 > ? ?type performance/io-threads > ? ?option thread-count 8 > ? ?subvolumes locks12 > end-volume > > volume posix13 > ?type storage/posix > ?option directory /mnt/data13 > end-volume > > volume locks13 > ? ?type features/locks > ? ?subvolumes posix13 > end-volume > > volume brick13 > ? ?type performance/io-threads > ? ?option thread-count 8 > ? ?subvolumes locks13 > end-volume > > volume posix14 > ?type storage/posix > ?option directory /mnt/data14 > end-volume > > volume locks14 > ? ?type features/locks > ? ?subvolumes posix14 > end-volume > > volume brick14 > ? ?type performance/io-threads > ? ?option thread-count 8 > ? ?subvolumes locks14 > end-volume > > volume posix15 > ?type storage/posix > ?option directory /mnt/data15 > end-volume > > volume locks15 > ? ?type features/locks > ? ?subvolumes posix15 > end-volume > > volume brick15 > ? ?type performance/io-threads > ? ?option thread-count 8 > ? ?subvolumes locks15 > end-volume > > volume posix16 > ?type storage/posix > ?option directory /mnt/data16 > end-volume > > volume locks16 > ? ?type features/locks > ? ?subvolumes posix16 > end-volume > > volume brick16 > ? ?type performance/io-threads > ? ?option thread-count 8 > ? ?subvolumes locks16 > end-volume > > volume posix17 > ?type storage/posix > ?option directory /mnt/data17 > end-volume > > volume locks17 > ? ?type features/locks > ? ?subvolumes posix17 > end-volume > > volume brick17 > ? ?type performance/io-threads > ? ?option thread-count 8 > ? ?subvolumes locks17 > end-volume > > volume posix18 > ?type storage/posix > ?option directory /mnt/data18 > end-volume > > volume locks18 > ? ?type features/locks > ? ?subvolumes posix18 > end-volume > > volume brick18 > ? ?type performance/io-threads > ? ?option thread-count 8 > ? ?subvolumes locks18 > end-volume > > volume posix19 > ?type storage/posix > ?option directory /mnt/data19 > end-volume > > volume locks19 > ? ?type features/locks > ? ?subvolumes posix19 > end-volume > > volume brick19 > ? ?type performance/io-threads > ? ?option thread-count 8 > ? ?subvolumes locks19 > end-volume > > volume posix20 > ?type storage/posix > ?option directory /mnt/data20 > end-volume > > volume locks20 > ? ?type features/locks > ? ?subvolumes posix20 > end-volume > > volume brick20 > ? ?type performance/io-threads > ? ?option thread-count 8 > ? ?subvolumes locks20 > end-volume > > volume posix21 > ?type storage/posix > ?option directory /mnt/data21 > end-volume > > volume locks21 > ? ?type features/locks > ? ?subvolumes posix21 > end-volume > > volume brick21 > ? ?type performance/io-threads > ? ?option thread-count 8 > ? ?subvolumes locks21 > end-volume > > volume posix22 > ?type storage/posix > ?option directory /mnt/data22 > end-volume > > volume locks22 > ? ?type features/locks > ? ?subvolumes posix22 > end-volume > > volume brick22 > ? ?type performance/io-threads > ? ?option thread-count 8 > ? ?subvolumes locks22 > end-volume > > volume posix23 > ?type storage/posix > ?option directory /mnt/data23 > end-volume > > volume locks23 > ? ?type features/locks > ? ?subvolumes posix23 > end-volume > > volume brick23 > ? ?type performance/io-threads > ? ?option thread-count 8 > ? ?subvolumes locks23 > end-volume > > volume posix24 > ?type storage/posix > ?option directory /mnt/data24 > end-volume > > volume locks24 > ? ?type features/locks > ? ?subvolumes posix24 > end-volume > > volume brick24 > ? ?type performance/io-threads > ? ?option thread-count 8 > ? ?subvolumes locks24 > end-volume > > volume server-tcp > ? ?type protocol/server > ? ?option transport-type tcp > ? ?option auth.addr.brick1.allow * > ? ?option auth.addr.brick2.allow * > ? ?option auth.addr.brick3.allow * > ? ?option auth.addr.brick4.allow * > ? ?option auth.addr.brick5.allow * > ? ?option auth.addr.brick6.allow * > ? ?option auth.addr.brick7.allow * > ? ?option auth.addr.brick8.allow * > ? ?option auth.addr.brick9.allow * > ? ?option auth.addr.brick10.allow * > ? ?option auth.addr.brick11.allow * > ? ?option auth.addr.brick12.allow * > ? ?option auth.addr.brick13.allow * > ? ?option auth.addr.brick14.allow * > ? ?option auth.addr.brick15.allow * > ? ?option auth.addr.brick16.allow * > ? ?option auth.addr.brick17.allow * > ? ?option auth.addr.brick18.allow * > ? ?option auth.addr.brick19.allow * > ? ?option auth.addr.brick20.allow * > ? ?option auth.addr.brick21.allow * > ? ?option auth.addr.brick22.allow * > ? ?option auth.addr.brick23.allow * > ? ?option auth.addr.brick24.allow * > ? ?option transport.socket.listen-port 6996 > ? ?option transport.socket.nodelay on > ? ?subvolumes brick1 brick2 brick3 brick4 brick5 brick6 brick7 brick8 > brick9 brick10 brick11 brick12 brick13 brick14 brick15 brick16 brick17 > brick18 brick19 brick20 brick21 brick22 brick23 brick24 > end-volume > -- http://philcryer.com