Hi, Le samedi 11 mai 2013 à 21:29 +0200, Pablo Neira Ayuso a écrit : > Hi Eric, > > On Fri, May 10, 2013 at 08:48:56AM +0200, Eric Leblond wrote: > > This patch adds an optional ring buffer option which modify > > the way database queries are made. The main thread is only handling > > kernel message reading and query formatting. The SQL request is made > > in a separate dedicated thread. > > The idea is to try to avoid buffer overrun by minimizing the time > > requested to treat kernel message. Doing synchronous SQL request, as > > it was made before was causing a delay which could cause some messages > > to be lost in case of burst from kernel side. > > Would be feasible to make asynchronous SQL requests instead, so you > can skip the use of pthread? That is an excellent question :) Here's a list of points. From the small to big one. From a performance point of view, ulogd will still be slow down by the request time due to the network communication with the db . I think the effect is really negligible but may have an impact in the case of distant db. From a lazy point of view, this would require to update all the database backends instead of simply updating the db abstraction layer as it is done with the proposed patch. Regarding the use of asynchronous request, I've always stop trying using them after reading the doc. So I've got no practical experience here and correct someone correct me if I'm wrong. What I've understood is that there is no "take my query and just do your work" function. Application has to get the result to the query and free them. For example, let's have a look at postgresql case. The doc on asynchronous API is here: http://www.postgresql.org/docs/6.4/static/libpq-chapter17044.htm Here's an interesting part of the documentation: PQsendQuery: Submit a query to Postgres without waiting for the result(s). TRUE is returned if the query was successfully dispatched, FALSE if not (in which case, use PQerrorMessage to get more information about the failure). After successfully calling PQsendQuery, call PQgetResult one or more times to obtain the query results. PQsendQuery may not be called again (on the same connection) until PQgetResult has returned NULL, indicating that the query is done. On other interesting point regarding PQgetResult: Don't forget to free each result object with PQclear when done with it. So, the modification would involve doing query with PQsendQuery and repeatedly call the PQgetResult function to get the result and be able to free it with PQclear. I think this system does not really fit with ulogd task which consists in multiple short queries. It is far more adapted for long queries and for GUI using them as ulogd will continuously be calling PQgetResult to free result it will not use. So, it is complicated and does not seem really adapted. Here's the points that make me consider using thread instead of asynchronous call. BR, -- Eric
Attachment:
signature.asc
Description: This is a digitally signed message part
