Thanks Rafi for workaround.
>> To find the root cause we need to get logs for the first failure of volume start or volume stop .
Below, exact steps to re-produce the issue and attached log file contents from /varlog/gluster folder(parsed)
STEPS to re-produce the issue
root@hostname:~# gluster volume create home-folder transport tcp,rdma storageN1:/gluster/disk1/home-
folder storageN2:/gluster/disk1/home- folder volume create: home-folder: success: please start the volume to access data
root@hostname:~#
root@hostname:~# gluster volume info home-folder
Volume Name: home-folder
Type: Distribute
Volume ID: 09abd02a-b760-459f-afde-
95b374eafc53 Status: Created
Snapshot Count: 0
Number of Bricks: 2
Transport-type: tcp,rdma
Bricks:
Brick1: storageN1:/gluster/disk1/home-
folder Brick2: storageN2:/gluster/disk1/home-
folder Options Reconfigured:
performance.readdir-ahead: on
nfs.disable: on
root@hostname:~# gluster volume status home-folder
Volume home-folder is not started
root@hostname:~# gluster volume start home-folder
volume start: home-folder: failed: Commit failed on localhost. Please check log file for details.
root@hostname:~# gluster volume status home-folder
Volume home-folder is not started
root@hostname:~# gluster volume start home-folder force
volume start: home-folder: success
root@hostname:~# gluster volume status home-folder
Status of volume: home-folder
Gluster process
TCP Port RDMA Port Online Pid ------------------------------
------------------------------ ------------------ Brick storageN1:/gluster/disk1/home-
folder N/A N/A Y 17737 Brick storageN2:/gluster/disk1/home-
folder N/A N/A N N/A
Task Status of Volume home-folder
------------------------------
------------------------------ ------------------ There are no active volume tasks
root@hostname:~# gluster volume info home-folder
Volume Name: home-folder
Type: Distribute
Volume ID: 09abd02a-b760-459f-afde-
95b374eafc53 Status: Started
Snapshot Count: 0
Number of Bricks: 2
Transport-type: tcp,rdma
Bricks:
Brick1: storageN1:/gluster/disk1/home-
folder Brick2: storageN2:/gluster/disk1/home-
folder Options Reconfigured:
performance.readdir-ahead: on
nfs.disable: on
root@hostname:~#
--
Deepak
From: Mohammed Rafi K C [mailto:rkavunga@xxxxxxxxxx]
Sent: Friday, February 24, 2017 1:08 AM
To: Deepak Naidu; gluster-users@xxxxxxxxxxx
Subject: Re: volume start: data0: failed: Commit failed on localhost.
It looks like it is ended up in split brain kind of situation. To find the root cause we need to get logs for the first failure of volume start or volume stop .
Or to work around it, you can do a volume start force.
Regards
Rafi KC
On 02/24/2017 01:36 PM, Deepak Naidu wrote:
I keep on getting this error when my config.transport is set to both tcp,rdma. The volume doesn’t start. I get the below error during volume start.
To get around this, I end up delete the volume, then configure either only rdma or tcp. May be I am missing something, just trying to get the volume up.
root@hostname:~# gluster volume start data0
volume start: data0: failed: Commit failed on localhost. Please check log file for details.
root@hostname:~#
root@ hostname:~# gluster volume status data0
Staging failed on storageN2. Error: Volume data0 is not started
root@ hostname:~
=============
[2017-02-24 08:00:29.923516] I [MSGID: 106499] [glusterd-handler.c:4349:__
glusterd_handle_status_volume] 0-management: Received status volume req for volume data0 [2017-02-24 08:00:29.926140] E [MSGID: 106153] [glusterd-syncop.c:113:gd_
collate_errors] 0-glusterd: Staging failed on storageN2. Error: Volume data0 is not started [2017-02-24 08:00:33.770505] I [MSGID: 106499] [glusterd-handler.c:4349:__
glusterd_handle_status_volume] 0-management: Received status volume req for volume data0 [2017-02-24 08:00:33.772824] E [MSGID: 106153] [glusterd-syncop.c:113:gd_
collate_errors] 0-glusterd: Staging failed on storageN2. Error: Volume data0 is not started =============
[2017-02-24 08:01:36.305165] E [MSGID: 106537] [glusterd-volume-ops.c:1660:
glusterd_op_stage_start_ volume] 0-management: Volume data0 already started [2017-02-24 08:01:36.305191] W [MSGID: 106122] [glusterd-mgmt.c:198:gd_mgmt_
v3_pre_validate_fn] 0-management: Volume start prevalidation failed. [2017-02-24 08:01:36.305198] E [MSGID: 106122] [glusterd-mgmt.c:884:glusterd_
mgmt_v3_pre_validate] 0-management: Pre Validation failed for operation Start on local node [2017-02-24 08:01:36.305205] E [MSGID: 106122] [glusterd-mgmt.c:2009:
glusterd_mgmt_v3_initiate_all_ phases] 0-management: Pre Validation Failed
--
Deepak
This email message is for the sole use of the intended recipient(s) and may contain confidential information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.
_______________________________________________ Gluster-users mailing listGluster-users@xxxxxxxxxxxhttp://lists.gluster.org/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://lists.gluster.org/mailman/listinfo/gluster-users
--
_______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://lists.gluster.org/mailman/listinfo/gluster-users
