Hi, From my code study it seems priv->ssl_ssl is not properly released, I made a patch and the glusterfsd memory leak is alleviated with my patch, but some otherwhere is still
leaking, I have no clue about the other leak points. --- a/rpc/rpc-transport/socket/src/socket.c +++ b/rpc/rpc-transport/socket/src/socket.c @@ -1019,7 +1019,16 @@ static void __socket_reset(rpc_transport_t *this) { memset(&priv->incoming, 0, sizeof(priv->incoming)); event_unregister_close(this->ctx->event_pool, priv->sock, priv->idx); - + if(priv->use_ssl&& priv->ssl_ssl) + { + gf_log(this->name, GF_LOG_TRACE, + "clear and reset for socket(%d), free ssl ", + priv->sock); + SSL_shutdown(priv->ssl_ssl); + SSL_clear(priv->ssl_ssl); + SSL_free(priv->ssl_ssl); + priv->ssl_ssl = NULL; + } priv->sock = -1; priv->idx = -1; priv->connected = -1; @@ -4238,6 +4250,16 @@ void fini(rpc_transport_t *this) { pthread_mutex_destroy(&priv->out_lock); pthread_mutex_destroy(&priv->cond_lock); pthread_cond_destroy(&priv->cond); + if(priv->use_ssl&& priv->ssl_ssl) + { + gf_log(this->name, GF_LOG_TRACE, + "clear and reset for socket(%d), free ssl ", + priv->sock); + SSL_shutdown(priv->ssl_ssl); + SSL_clear(priv->ssl_ssl); + SSL_free(priv->ssl_ssl); + priv->ssl_ssl = NULL; + } if (priv->ssl_private_key) { GF_FREE(priv->ssl_private_key); } From: Zhou, Cynthia (NSB - CN/Hangzhou)
We scan it use memory-leak tool, there are following prints. We doubt some open ssl lib malloc is is not properly freed by glusterfs code. er+0x2af [libglusterfs.so.0.0.1]\n\t\tstart_thread+0xda
[libpthread-2.27.so]' cynthia From: Zhou, Cynthia (NSB - CN/Hangzhou)
I’ve test on glusterfs3.12.15 and glusterfs5.5 all have this issue, after enable tls ssl socket,
when execute gluster v heal <vol-name> info, will trigger glfshel to connect glusterfsd process, and cause glusterfsd process memory leak. Could you please try in your env?
cynthia From: Atin Mukherjee <amukherj@xxxxxxxxxx>
On Wed, 17 Apr 2019 at 10:53, Zhou, Cynthia (NSB - CN/Hangzhou) <cynthia.zhou@xxxxxxxxxxxxxxx> wrote:
What gluster version are you testing? Would you be able to continue your investigation and share the root cause? -- - Atin (atinm) |
_______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx https://lists.gluster.org/mailman/listinfo/gluster-devel