Re: Regression Failure: ./tests/basic/quota.t

[Raghavendra Talur <raghavendra.talur@xxxxxxxxx>]

On Thu, Jul 2, 2015 at 1:26 PM, Nithya Balachandran <nbalacha@xxxxxxxxxx> wrote:

----- Original Message -----
> From: "Kotresh Hiremath Ravishankar" <khiremat@xxxxxxxxxx>
> To: "Susant Palai" <spalai@xxxxxxxxxx>
> Cc: "Gluster Devel" <gluster-devel@xxxxxxxxxxx>

> Sent: Thursday, July 2, 2015 1:03:18 PM
> Subject: Re: Regression Failure: ./tests/basic/quota.t
>
> Comments inline.
>
> Thanks and Regards,
> Kotresh H R
>
> ----- Original Message -----
> > From: "Susant Palai" <spalai@xxxxxxxxxx>
> > To: "Sachin Pandit" <spandit@xxxxxxxxxx>
> > Cc: "Kotresh Hiremath Ravishankar" <khiremat@xxxxxxxxxx>, "Gluster Devel"
> > <gluster-devel@xxxxxxxxxxx>
> > Sent: Thursday, July 2, 2015 12:35:08 PM
> > Subject: Re: Regression Failure: ./tests/basic/quota.t
> >
> > Comments inline.
> >
> > ----- Original Message -----
> > > From: "Sachin Pandit" <spandit@xxxxxxxxxx>
> > > To: "Kotresh Hiremath Ravishankar" <khiremat@xxxxxxxxxx>
> > > Cc: "Gluster Devel" <gluster-devel@xxxxxxxxxxx>
> > > Sent: Thursday, July 2, 2015 12:21:44 PM
> > > Subject: Re: Regression Failure: ./tests/basic/quota.t
> > >
> > > ----- Original Message -----
> > > > From: "Vijaikumar M" <vmallika@xxxxxxxxxx>
> > > > To: "Kotresh Hiremath Ravishankar" <khiremat@xxxxxxxxxx>, "Gluster
> > > > Devel"
> > > > <gluster-devel@xxxxxxxxxxx>
> > > > Cc: "Sachin Pandit" <spandit@xxxxxxxxxx>
> > > > Sent: Thursd, TOTAL CHANGELOGS: 106
> [2015-07-02 07:01:06.883504] E
> [gf-history-changelog.c:877:gf_history_changelog] 0-gfchangelog: wrong
> result for start: 1435818ay, July 2, 2015 12:01:03 PM
> > > > Subject: Re: Regression Failure: ./tests/basic/quota.t
> > > >
> > > > We look into this issue
> > > >
> > > > Thanks,
> > > > Vijay
> > > >
> > > > On Thursday 02 July 2015 11:46 AM, Kotresh Hiremath Ravishankar wrote:
> > > > > Hi,
> > > > >
> > > > > I see quota.t regression failure for the following. The changes are
> > > > > related
> > > > > to
> > > > > example programs in libgfchangelog.
> > > > >
> > > > > http://build.gluster.org/job/rackspace-regression-2GB-triggered/11785/consoleFull
> > > > >
> > > > > Could someone from quota team, take a look at it.
> > >
> > > Hi,
> > >
> > > I had a quick look at this. It looks like the following test case failed
> > >
> > > TEST $CLI volume add-brick $V0 $H0:$B0/brick{3,4}
> > > EXPECT_WITHIN $REBALANCE_TIMEOUT "0" rebalance_completed
> > >

Looks like the same "port in use" issue. From the d-backends-brick3.log:


[2015-07-01 09:27:17.821430] E [socket.c:818:__socket_server_bind] 0-tcp.patchy-server: binding to  failed: Address already in use
[2015-07-01 09:27:17.821441] E [socket.c:821:__socket_server_bind] 0-tcp.patchy-server: Port is already in use
[2015-07-01 09:27:17.821452] W [rpcsvc.c:1599:rpcsvc_transport_create] 0-rpc-service: listening on transport failed
[2015-07-01 09:27:17.821462] W [MSGID: 115045] [server.c:996:init] 0-patchy-server: creation of listener failed
[2015-07-01 09:27:17.821475] E [MSGID: 101019] [xlator.c:423:xlator_init] 0-patchy-server: Initialization of volume 'patchy-server' failed, review your volfile again
[2015-07-01 09:27:17.821485] E [MSGID: 101066] [graph.c:323:glusterfs_graph_init] 0-patchy-server: initializing translator failed
[2015-07-01 09:27:17.821495] E [MSGID: 101176] [graph.c:669:glusterfs_graph_activate] 0-graph: init failed
[2015-07-01 09:27:17.821891] W [glusterfsd.c:1214:cleanup_and_exit] (--> 0-: received signum (0), shutting down

The patch which exposed this bug is being reverted till the underlying bug is also fixed.

You can monitor revert patches here 

master: http://review.gluster.org/11507

3.7 branch: http://review.gluster.org/11508

Please rebase your patches after the above patches are merged to ensure that you patches pass regression.

 

> > >
> > > I looked at the logs too, and found out the following errors
> > >
> > > patchy-rebalance.log:[2015-07-01 09:27:23.040756] E [MSGID: 109026]
> > > [dht-rebalance.c:2689:gf_defrag_start_crawl] 0-patchy-dht: fix layout on
> > > /
> > > failed
> > > build-install-etc-glusterfs-glusterd.vol.log:[2015-07-01 09:27:23.040998]
> > > E
> > > [MSGID: 106224]
> > > [glusterd-rebalance.c:960:glusterd_defrag_event_notify_handle]
> > > 0-management:
> > > Failed to update status
> > > StartMigrationDuringRebalanceTest-rebalance.log:[2015-06-19
> > > 14:34:47.557887]
> > > E [rpc-clnt.c:362:saved_frames_unwind] (-->
> > > /build/install/lib/libglusterfs.so.0(_gf_log_callingfn+0x240)[0x7fc882d04d5a]
> > > (-->
> > > /build/install/lib/libgfrpc.so.0(saved_frames_unwind+0x212)[0x7fc882ace086]
> > > (-->
> > > /build/install/lib/libgfrpc.so.0(saved_frames_destroy+0x1f)[0x7fc882ace183]
> > > (-->
> > > /build/install/lib/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x11e)[0x7fc882ace615]
> > > (-->
> > > /build/install/lib/libgfrpc.so.0(rpc_clnt_notify+0x147)[0x7fc882acf00f]
> > > ))))) 0-StartMigrationDuringRebalanceTest-client-0: forced unwinding
> > > frame
> > > type(GlusterFS 3.3) op(LOOKUP(27)) called at 2015-06-19 14:34:47.554862
> > > (xid=0xc)
> > > StartMigrationDuringRebalanceTest-rebalance.log:[2015-06-19
> > > 14:34:47.561191]
> > > E [MSGID: 114031] [client-rpc-fops.c:1623:client3_3_inodelk_cbk]
> > > 0-StartMigrationDuringRebalanceTest-client-0: remote operation failed:
> > > Transport endpoint is not connected [Transport endpoint is not connected]
> > > StartMigrationDuringRebalanceTest-rebalance.log:[2015-06-19
> > > 14:34:47.561417]
> > > E [socket.c:2332:socket_connect_finish]
> > > 0-StartMigrationDuringRebalanceTest-client-0: connection to
> > > 23.253.62.104:24007 failed (Connection refused)
> > > StartMigrationDuringRebalanceTest-rebalance.log:[2015-06-19
> > > 14:34:47.561707]
> > > E [dht-common.c:2643:dht_find_local_subvol_cbk]
> > > 0-StartMigrationDuringRebalanceTest-dht: getxattr err (Transport endpoint
> > > is
> > > not connected) for dir
> > >
> > Seems like a network partition. Rebalance fails if there it receives
> > ENOTCONN
> > on it's child.
>
> Is this intended to happen on regression machines?
> >
> > >
> > > Any help regarding this or more information on this would be much
> > > appreciated.
> > >
> > > Thanks,
> > > Sachin Pandit.
> > >
> > >
> > > > >
> > > > > Thanks and Regards,
> > > > > Kotresh H R
> > > > >
> > > >
> > > >
> > > _______________________________________________
> > > Gluster-devel mailing list
> > > Gluster-devel@xxxxxxxxxxx
> > > http://www.gluster.org/mailman/listinfo/gluster-devel
> > >
> >
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel@xxxxxxxxxxx
> http://www.gluster.org/mailman/listinfo/gluster-devel
>
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-devel

--

Raghavendra Talur 

_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-devel