Hi Soumya,
Can you send a fix to this regression on upstream master too? This patch is merged there.
regards,
Raghavendra
On Tue, Mar 1, 2016 at 10:34 PM, Kotresh Hiremath Ravishankar <khiremat@xxxxxxxxxx> wrote:
Hi Soumya,
I analysed the issue and found out that crash has happened because of the patch [1].
The patch doesn't set transport object to NULL in 'rpc_clnt_disable' but instead does it on
'rpc_clnt_trigger_destroy'. So if there are pending rpc invocations on the rpc object that
is disabled (those instances are possible as happening now in changelog), it will trigger a
CONNECT notify again with 'mydata' that is freed causing a crash. This happens because
'rpc_clnt_submit' reconnects if rpc is not connected.
rpc_clnt_submit (...) {
...
if (conn->connected == 0) {
ret = rpc_transport_connect (conn->trans,
conn->config.remote_port);
}
...
}
Without your patch, conn->trans was set NULL and hence CONNECT fails not resulting with
CONNECT notify call. And also the cleanup happens in failure path.
So the memory leak can happen, if there is no try for rpc invocation after DISCONNECT.
It will be cleaned up otherwise.
[1] http://review.gluster.org/#/c/13507/
Thanks and Regards,
Kotresh H R
----- Original Message -----
> From: "Kotresh Hiremath Ravishankar" <khiremat@xxxxxxxxxx>
> To: "Soumya Koduri" <skoduri@xxxxxxxxxx>
> Cc: avishwan@xxxxxxxxxx, "Gluster Devel" <gluster-devel@xxxxxxxxxxx>
> Sent: Monday, February 29, 2016 4:15:22 PM
> Subject: Re: Cores generated with ./tests/geo-rep/georep-basic-dr-tarssh.t
>
> Hi Soumya,
>
> I just tested that it is reproducible only with your patch both in master and
> 3.76 branch.
> The geo-rep test cases are marked bad in master. So it's not hit in master.
> rpc is introduced
> in changelog xlator to communicate to applications via libgfchangelog.
> Venky/Me will check
> why is the crash happening and will update.
>
>
> Thanks and Regards,
> Kotresh H R
>
> ----- Original Message -----
> > From: "Soumya Koduri" <skoduri@xxxxxxxxxx>
> > To: avishwan@xxxxxxxxxx, "kotresh" <khiremat@xxxxxxxxxx>
> > Cc: "Gluster Devel" <gluster-devel@xxxxxxxxxxx>
> > Sent: Monday, February 29, 2016 2:10:51 PM
> > Subject: Cores generated with ./tests/geo-rep/georep-basic-dr-tarssh.t
> >
> > Hi Aravinda/Kotresh,
> >
> > With [1], I consistently see cores generated with the test
> > './tests/geo-rep/georep-basic-dr-tarssh.t' in release-3.7 branch. From
> > the cores, looks like we are trying to dereference a freed
> > changelog_rpc_clnt_t(crpc) object in changelog_rpc_notify(). Strangely
> > this was not reported in master branch.
> >
> > I tried debugging but couldn't find any possible suspects. I request you
> > to take a look and let me know if [1] caused any regression.
> >
> > Thanks,
> > Soumya
> >
> > [1] http://review.gluster.org/#/c/13507/
> >
>
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-devel
Raghavendra G
_______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-devel
