Re: cannot start a gluster volume

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Maybe remove peer glusterp3 via "gluster peer detach" then re add it?

On 14 October 2016 at 12:16, Thing <thing.thing@xxxxxxxxx> wrote:
> I seem to have a broken volume on glusterp3 which I odnt seem to be able to
> fix, how to please?
>
> ========
> [root@glusterp1 /]# ls -l /data1
> total 4
> -rw-r--r--. 2 root root 0 Dec 14  2015 file1
> -rw-r--r--. 2 root root 0 Dec 14  2015 file2
> -rw-r--r--. 2 root root 0 Dec 14  2015 file3
> -rw-r--r--. 2 root root 0 Dec 14  2015 file.ipa1
> [root@glusterp1 /]# gluster volume status
> Staging failed on glusterp3.graywitch.co.nz. Error: Volume volume1 does not
> exist
>
> [root@glusterp1 /]# gluster
> gluster> volume info
>
> Volume Name: volume1
> Type: Replicate
> Volume ID: 91eef74e-4016-4bbe-8e86-01c88c64593f
> Status: Started
> Snapshot Count: 0
> Number of Bricks: 1 x 3 = 3
> Transport-type: tcp
> Bricks:
> Brick1: glusterp1.graywitch.co.nz:/data1
> Brick2: glusterp2.graywitch.co.nz:/data1
> Brick3: glusterp3.graywitch.co.nz:/data1
> Options Reconfigured:
> performance.readdir-ahead: on
> gluster> exit
> [root@glusterp1 /]# gluster volume heal volume1 info
> Brick glusterp1.graywitch.co.nz:/data1
> Status: Connected
> Number of entries: 0
>
> Brick glusterp2.graywitch.co.nz:/data1
> Status: Connected
> Number of entries: 0
>
> Brick glusterp3.graywitch.co.nz:/data1
> Status: Connected
> Number of entries: 0
>
> [root@glusterp1 /]# gluster volume info
>
> Volume Name: volume1
> Type: Replicate
> Volume ID: 91eef74e-4016-4bbe-8e86-01c88c64593f
> Status: Started
> Snapshot Count: 0
> Number of Bricks: 1 x 3 = 3
> Transport-type: tcp
> Bricks:
> Brick1: glusterp1.graywitch.co.nz:/data1
> Brick2: glusterp2.graywitch.co.nz:/data1
> Brick3: glusterp3.graywitch.co.nz:/data1
> Options Reconfigured:
> performance.readdir-ahead: on
> [root@glusterp1 /]# gluster volume heal volume1 full
> Launching heal operation to perform full self heal on volume volume1 has
> been unsuccessful on bricks that are down. Please check if all brick
> processes are running.
> [root@glusterp1 /]#
> =============
>
> On 14 October 2016 at 12:40, Thing <thing.thing@xxxxxxxxx> wrote:
>>
>> So glusterp3 is in a reject state,
>>
>> [root@glusterp1 /]# gluster peer status
>> Number of Peers: 2
>>
>> Hostname: glusterp2.graywitch.co.nz
>> Uuid: 93eebe2c-9564-4bb0-975f-2db49f12058b
>> State: Peer in Cluster (Connected)
>> Other names:
>> glusterp2
>>
>> Hostname: glusterp3.graywitch.co.nz
>> Uuid: 5d59b704-e42f-46c6-8c14-cf052c489292
>> State: Peer Rejected (Connected)
>> Other names:
>> glusterp3
>> [root@glusterp1 /]#
>>
>> ========
>>
>> [root@glusterp2 /]# gluster peer status
>> Number of Peers: 2
>>
>> Hostname: glusterp1.graywitch.co.nz
>> Uuid: 4ece8509-033e-48d1-809f-2079345caea2
>> State: Peer in Cluster (Connected)
>> Other names:
>> glusterp1
>>
>> Hostname: glusterp3.graywitch.co.nz
>> Uuid: 5d59b704-e42f-46c6-8c14-cf052c489292
>> State: Peer Rejected (Connected)
>> Other names:
>> glusterp3
>> [root@glusterp2 /]#
>>
>> ========
>>
>> [root@glusterp3 /]# gluster peer status
>> Number of Peers: 2
>>
>> Hostname: glusterp1.graywitch.co.nz
>> Uuid: 4ece8509-033e-48d1-809f-2079345caea2
>> State: Peer Rejected (Connected)
>> Other names:
>> glusterp1
>>
>> Hostname: glusterp2.graywitch.co.nz
>> Uuid: 93eebe2c-9564-4bb0-975f-2db49f12058b
>> State: Peer Rejected (Connected)
>> Other names:
>> glusterp2
>>
>> ==========
>> on glusterp3 gluster is dead and will not start,
>>
>> [root@glusterp3 /]# systemctl status gluster
>> ● gluster.service
>>    Loaded: not-found (Reason: No such file or directory)
>>    Active: inactive (dead)
>>
>> [root@glusterp3 /]# systemctl restart gluster
>> Failed to restart gluster.service: Unit gluster.service failed to load: No
>> such file or directory.
>> [root@glusterp3 /]# systemctl enable gluster
>> Failed to execute operation: Access denied
>> [root@glusterp3 /]# systemctl enable gluster.service
>> Failed to execute operation: Access denied
>> [root@glusterp3 /]# systemctl start gluster.service
>> Failed to start gluster.service: Unit gluster.service failed to load: No
>> such file or directory.
>>
>> ==========
>>
>> [root@glusterp3 /]# rpm -qa |grep gluster
>> glusterfs-client-xlators-3.8.4-1.el7.x86_64
>> glusterfs-server-3.8.4-1.el7.x86_64
>> nfs-ganesha-gluster-2.3.3-1.el7.x86_64
>> glusterfs-cli-3.8.4-1.el7.x86_64
>> glusterfs-api-3.8.4-1.el7.x86_64
>> glusterfs-fuse-3.8.4-1.el7.x86_64
>> glusterfs-ganesha-3.8.4-1.el7.x86_64
>> glusterfs-3.8.4-1.el7.x86_64
>> centos-release-gluster38-1.0-1.el7.centos.noarch
>> glusterfs-libs-3.8.4-1.el7.x86_64
>> [root@glusterp3 /]#
>>
>> ?
>>
>> On 14 October 2016 at 12:31, Thing <thing.thing@xxxxxxxxx> wrote:
>>>
>>> Hmm seem I have something rather inconsistent,
>>>
>>> [root@glusterp1 /]# gluster volume create gv1 replica 3
>>> glusterp1:/brick1/gv1 glusterp2:/brick1/gv1 glusterp3:/brick1/gv1
>>> volume create: gv1: failed: Host glusterp3 is not in 'Peer in Cluster'
>>> state
>>> [root@glusterp1 /]# gluster peer probe glusterp3
>>> peer probe: success. Host glusterp3 port 24007 already in peer list
>>> [root@glusterp1 /]# gluster peer probe glusterp2
>>> peer probe: success. Host glusterp2 port 24007 already in peer list
>>> [root@glusterp1 /]# gluster volume create gv1 replica 3
>>> glusterp1:/brick1/gv1 glusterp2:/brick1/gv1 glusterp3:/brick1/gv1
>>> volume create: gv1: failed: /brick1/gv1 is already part of a volume
>>> [root@glusterp1 /]# gluster volume show
>>> unrecognized word: show (position 1)
>>> [root@glusterp1 /]# gluster volume
>>> add-brick        delete           info             quota            reset
>>> status
>>> barrier          geo-replication  list             rebalance        set
>>> stop
>>> clear-locks      heal             log              remove-brick     start
>>> sync
>>> create           help             profile          replace-brick
>>> statedump        top
>>> [root@glusterp1 /]# gluster volume list
>>> volume1
>>> [root@glusterp1 /]# gluster volume start gv0
>>> volume start: gv0: failed: Volume gv0 does not exist
>>> [root@glusterp1 /]# gluster volume start gv1
>>> volume start: gv1: failed: Volume gv1 does not exist
>>> [root@glusterp1 /]# gluster volume status
>>> Status of volume: volume1
>>> Gluster process                             TCP Port  RDMA Port  Online
>>> Pid
>>>
>>> ------------------------------------------------------------------------------
>>> Brick glusterp1.graywitch.co.nz:/data1      49152     0          Y
>>> 2958
>>> Brick glusterp2.graywitch.co.nz:/data1      49152     0          Y
>>> 2668
>>> NFS Server on localhost                     N/A       N/A        N
>>> N/A
>>> Self-heal Daemon on localhost               N/A       N/A        Y
>>> 1038
>>> NFS Server on glusterp2.graywitch.co.nz     N/A       N/A        N
>>> N/A
>>> Self-heal Daemon on glusterp2.graywitch.co.
>>> nz                                          N/A       N/A        Y
>>> 676
>>>
>>> Task Status of Volume volume1
>>>
>>> ------------------------------------------------------------------------------
>>> There are no active volume tasks
>>>
>>> [root@glusterp1 /]#
>>>
>>> On 14 October 2016 at 12:20, Thing <thing.thing@xxxxxxxxx> wrote:
>>>>
>>>> I deleted a gluster volume gv0 as I wanted to make it thin provisioned.
>>>>
>>>> I have rebuilt "gv0" but I am getting a failure,
>>>>
>>>> ==========
>>>> [root@glusterp1 /]# df -h
>>>> Filesystem                     Size  Used Avail Use% Mounted on
>>>> /dev/mapper/centos-root         20G  3.9G   17G  20% /
>>>> devtmpfs                       1.8G     0  1.8G   0% /dev
>>>> tmpfs                          1.8G   12K  1.8G   1% /dev/shm
>>>> tmpfs                          1.8G  8.9M  1.8G   1% /run
>>>> tmpfs                          1.8G     0  1.8G   0% /sys/fs/cgroup
>>>> /dev/mapper/centos-tmp         3.9G   33M  3.9G   1% /tmp
>>>> /dev/mapper/centos-home         50G   41M   50G   1% /home
>>>> /dev/mapper/centos-data1       120G   33M  120G   1% /data1
>>>> /dev/sda1                      997M  312M  685M  32% /boot
>>>> /dev/mapper/centos-var          20G  401M   20G   2% /var
>>>> tmpfs                          368M     0  368M   0% /run/user/1000
>>>> /dev/mapper/vol_brick1-brick1  100G   33M  100G   1% /brick1
>>>> [root@glusterp1 /]# mkdir /brick1/gv0
>>>> [root@glusterp1 /]# gluster volume create gv0 replica 3
>>>> glusterp1:/brick1/gv0 glusterp2:/brick1/gv0 glusterp3:/brick1/gv0
>>>> volume create: gv0: failed: Host glusterp3 is not in 'Peer in Cluster'
>>>> state
>>>> [root@glusterp1 /]# gluster peer probe glusterp3
>>>> peer probe: success. Host glusterp3 port 24007 already in peer list
>>>> [root@glusterp1 /]# gluster volume create gv0 replica 3
>>>> glusterp1:/brick1/gv0 glusterp2:/brick1/gv0 glusterp3:/brick1/gv0
>>>> volume create: gv0: failed: /brick1/gv0 is already part of a volume
>>>> [root@glusterp1 /]# gluster volume start gv0
>>>> volume start: gv0: failed: Volume gv0 does not exist
>>>> [root@glusterp1 /]# gluster volume create gv0 replica 3
>>>> glusterp1:/brick1/gv0 glusterp2:/brick1/gv0 glusterp3:/brick1/gv0 --force
>>>> unrecognized option --force
>>>> [root@glusterp1 /]# gluster volume create gv0 replica 3
>>>> glusterp1:/brick1/gv0 glusterp2:/brick1/gv0 glusterp3:/brick1/gv0
>>>> volume create: gv0: failed: /brick1/gv0 is already part of a volume
>>>> [root@glusterp1 /]#
>>>> ==========
>>>>
>>>> Obviously something isnt happy here but I have no idea what.......
>>>>
>>>> how to fix this please?
>>>
>>>
>>
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users@xxxxxxxxxxx
> http://www.gluster.org/mailman/listinfo/gluster-users



-- 
Lindsay
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users




[Index of Archives]     [Gluster Development]     [Linux Filesytems Development]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux