On Wed, Feb 22, 2023 at 8:35 AM Michael Arnold <myk321@xxxxxxxxx> wrote:
>
> Am looking for guidance on how to fix a memory leak when using libpq PQExecParams(). Memory leaks through CRYPTO_zalloc() and arises when using json_agg(). None-JSON based PQExecParams() calls are not leaking.
>
> Using Postgresql 13.6 (Ubuntu 13.6-0ubuntu0.21.10.1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 11.2.0-7ubuntu2) 11.2.0, 64-bit
> Accessed by a c/c++ application via libpq (libssl3 and libcrypt are also linked)
>
> Valgrind reports:
>
> ==4107== 2,712 bytes in 3 blocks are definitely lost in loss record 265 of 276
> ==4107== at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==4107== by 0x4B1F41D: CRYPTO_zalloc (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
> ==4107== by 0x4AD50A8: ??? (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
> ==4107== by 0x4AD51EC: ERR_clear_error (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
> ==4107== by 0x488D52A: ??? (in /usr/lib/x86_64-linux-gnu/libpq.so.5.14)
> ==4107== by 0x488E89D: ??? (in /usr/lib/x86_64-linux-gnu/libpq.so.5.14)
> ==4107== by 0x488E923: PQsendQueryParams (in /usr/lib/x86_64-linux-gnu/libpq.so.5.14)
> ==4107== by 0x4893C27: PQexecParams (in /usr/lib/x86_64-linux-gnu/libpq.so.5.14)
> ==4107== by 0x3E1613: getJSON(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) (dbInterface.cpp:7202)
> ==4107== by 0x47ACEC: tickerHandler::handleRequest(Poco::Net::HTTPServerRequest&, Poco::Net::HTTPServerResponse&) (externalInterface.cpp:615)
> ==4107== by 0x5096E18: Poco::Net::HTTPServerConnection::run() (in /usr/lib/x86_64-linux-gnu/libPocoNet.so.80)
> ==4107== by 0x50D935A: Poco::Net::TCPServerConnection::start() (in /usr/lib/x86_64-linux-gnu/libPocoNet.so.80)
>
> i.e. call to PQExecParams() within getJSON() is leaking via CRYTO_zalloc(). The code for getJSON() is reproduced below.
>
> std::string getJSON(std::string sqlQuery){
> PGconn *dbConn = nullptr;
> int nParams = 0;
> int resultFormat = 1; //binary format
> size_t noRecords = 0;
> PGresult *res = nullptr;
> std::string resultJSON;
>
> std::string fullQuery = "select json_agg(t) FROM (" + sqlQuery + ") t;";
>
> if ((dbConn = getConnection(connectionPool)) != nullptr) {
> res = PQexecParams(dbConn,
> fullQuery.c_str(),
> nParams,
> NULL,
> NULL,
> NULL,
> NULL, resultFormat);
>
> //Check if there was a problem
> if (PQresultStatus(res) != PGRES_TUPLES_OK) {
> if (dbConn != nullptr){
> returnConnection(connectionPool, dbConn);
> PQclear(res);
> }
> return (resultJSON);
> } //End of result checking
>
> //If we successfully get the information then populate resultJSON
> noRecords = (int64_t) PQntuples(res);
> if (noRecords == 1)
> resultJSON = std::string((const char*) PQgetvalue(res, 0, 0));
>
> //Clean-up
> PQclear(res);
> returnConnection(connectionPool, dbConn);
> }
>
> //select json_arr may return an empty string "", but this is not valid json and json.parse will error
> // to avoid this return a valid json empty string
> if (resultJSON.empty())
> resultJSON = "\"\"";
>
> return (resultJSON);
> }
>
> getConnection() and returnConnection() are application specific calls to get and replace valid PGconn* to / from an in-app postgres connection pool.
>
> Setup is really basic:
> 1. In postgresql.conf
> 1a. Uncomment listen_addresses
> 1b. Replace localhost with *
> 1c. Uncomment password_encryption
>
> 2. In pg_hba.conf
> 2a. Add the local network:
> host all all 192.168.0.101/24 md5
> 2b. Replace ident with md5
Before you do anything with Valgrind, you should rebuild Postgres,
your program, and dependent libraries of interest with -g -O1.
Otherwise you risk wasting [a lot] time on false positives.
Also see https://valgrind.org/docs/manual/quick-start.html .
Jeff
