On 13/02/14 12:39, Vijay Bellur wrote:
On 02/11/2014 08:48 PM, tegner@xxxxxxxxx wrote:
Hi,
have a system consisting of 4 bricks, distributed, 3.4.1, and I have
noticed that some of the files are stored on three of the bricks.
Typically a listing can look something like this:
brick1: ---------T 2 root root 0 Feb 11 15:47 /mnt/raid6/file
brick2: -rw------- 2 2686 2022 10545 Mar 6 2012 /mnt/raid6/file
brick3: -rw------- 2 2686 2022 10545 Mar 6 2012 /mnt/raid6/file
brick4: no such file or directory
Can you please post your volume configuration?
gluster volume info gives:
**************************************************************
Volume Name: glusterKumiko
Type: Distribute
Volume ID: f54efdc7-0664-409d-b3bc-c39f31040f5e
Status: Started
Number of Bricks: 4
Transport-type: tcp
Bricks:
Brick1: kumiko01IB:/mnt/raid6
Brick2: kumiko02IB:/mnt/raid6
Brick3: kumiko03IB:/mnt/raid6
Brick4: kumiko04IB:/mnt/raid6
**********************************************************************
Also, the file /etc/glusterfs/glusterd.vol have the following content:
**********************************************************************
volume management
type mgmt/glusterd
option working-directory /var/lib/glusterd
option transport-type socket,rdma
option transport.socket.keepalive-time 10
option transport.socket.keepalive-interval 2
option transport.socket.read-fail-log off
# option base-port 49152
end-volume
***********************************************************************
Should I be worried that rdma is listed as transport-type (and the lack
of tcp)?
There are quite a few of these files, and it would be tedious to clean
them up manually. Is there a way to have gluster fix these?
This is not a normal occurrence. Do you happen to know the sequence of
steps which resulted in this?
Probably messed up:
1. When the volume was created (with RDMA) two of the bricks were
populated with files I wanted to "merge into" gluster. It seemed as if
they became "incorporated" after doing a "rsync -vaun" from our primary
file system (the system in question is used as a mirror).
2. At a later time I wanted to go away from RDMA (using IPoIB instead),
and at that time I removed the old volume, followed hints from
http://joejulian.name/blog/glusterfs-path-or-a-prefix-of-it-is-already-part-of-a-volume/
and created the new volume (with tcp instead of rdma).
3. Just recently I upgraded to 3.4.2-1, and started a
gluster volume rebalance glusterKumiko fix-layout start (since I have a
lot of
"disk layout missing" and "mismatching layouts" in the logs).
Regards, and thanks!
/jon
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://supercolony.gluster.org/mailman/listinfo/gluster-users
