On 02/02/2010 11:54 AM, Daniel P. Berrange wrote:
> On Tue, Jan 12, 2010 at 03:26:26PM -0500, Cole Robinson wrote:
>> This allows debug statements and raised errors in hook functions to
>> actually be logged somewhere (stderr). Users can enable debugging in the
>> daemon and now see more info in /var/log/libvirt/...
>>
>> Signed-off-by: Cole Robinson <crobinso@xxxxxxxxxx>
>> ---
>> src/util/util.c | 6 ++++++
>> 1 files changed, 6 insertions(+), 0 deletions(-)
>>
>> diff --git a/src/util/util.c b/src/util/util.c
>> index 44a4b2f..23d781d 100644
>> --- a/src/util/util.c
>> +++ b/src/util/util.c
>> @@ -334,6 +334,7 @@ __virExec(virConnectPtr conn,
>> int pipeerr[2] = {-1,-1};
>> int childout = -1;
>> int childerr = -1;
>> + int logprio;
>> sigset_t oldmask, newmask;
>> struct sigaction sig_action;
>>
>> @@ -452,6 +453,11 @@ __virExec(virConnectPtr conn,
>> of being seen / logged */
>> virSetErrorFunc(NULL, NULL);
>>
>> + /* Make sure any hook logging is sent to stderr */
>> + logprio = virLogGetDefaultPriority();
>> + virLogReset();
>> + virLogSetDefaultPriority(logprio);
>> +
>
> This patch turns out to cause a deadlock in libvirtd. The problem is
> that fork() preserves the state of any mutexes which are locked.
>
> So if some thread in libvirtd is currently executing a logging call,
> while another thread calls virExec(), that other thread no longer
> exists in the child, but its lock is never released. So when the
> child then does virLogReset() it deadlocks.
>
> I'm actuall surprised we've not hit this problem before, since any call
> to virRaiseError in the child will also eventually run a logging function
> which will in turn acquire a lock.
>
> The only way I see to address this, is for the parent process to call
> virLogLock(), immediately before fork(), and then virLogUnlock()
> afterwards in both parent & child. This will ensure that no other thread
> can be holding the lock across fork().
>
> https://bugzilla.redhat.com/show_bug.cgi?id=561066
>
Does this look okay? I haven't tested that it fixes the issue (though I
haven't reproduced the error either).
- Cole
diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
index e5e8860..7573af3 100644
--- a/src/libvirt_private.syms
+++ b/src/libvirt_private.syms
@@ -366,6 +366,8 @@ virLogParseOutputs;
virLogStartup;
virLogShutdown;
virLogReset;
+virLogLock;
+virLogUnlock;
# memory.h
diff --git a/src/util/logging.c b/src/util/logging.c
index 3b3c309..11a4b06 100644
--- a/src/util/logging.c
+++ b/src/util/logging.c
@@ -133,11 +133,11 @@ static int virLogResetOutputs(void);
*/
virMutex virLogMutex;
-static void virLogLock(void)
+void virLogLock(void)
{
virMutexLock(&virLogMutex);
}
-static void virLogUnlock(void)
+void virLogUnlock(void)
{
virMutexUnlock(&virLogMutex);
}
diff --git a/src/util/logging.h b/src/util/logging.h
index 49e2f6d..5f61f59 100644
--- a/src/util/logging.h
+++ b/src/util/logging.h
@@ -128,6 +128,8 @@ extern int virLogDefineOutput(virLogOutputFunc f, virLogCloseFunc c, void *data,
* Internal logging API
*/
+extern void virLogLock(void);
+extern void virLogUnlock(void);
extern int virLogStartup(void);
extern int virLogReset(void);
extern void virLogShutdown(void);
diff --git a/src/util/util.c b/src/util/util.c
index cf1290d..901c0d2 100644
--- a/src/util/util.c
+++ b/src/util/util.c
@@ -415,12 +415,19 @@ __virExec(virConnectPtr conn,
childerr = null;
}
+ /* Ensure we hold the logging lock, to protect child processes
+ * from deadlocking on another threads inheirited mutex state */
+ virLogLock();
+
if ((pid = fork()) < 0) {
virReportSystemError(conn, errno,
"%s", _("cannot fork child process"));
goto cleanup;
}
+ /* Unlock for both parent and child process */
+ virLogUnlock();
+
if (pid) { /* parent */
close(null);
if (outfd && *outfd == -1) {
--
libvir-list mailing list
libvir-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvir-list