On (09/04/19 16:42), Qian Cai wrote: > > Let me think more. > > To summary, those look to me are all good long-term improvement that would > reduce the likelihood of this kind of livelock in general especially for other > unknown allocations that happen while processing softirqs, but it is still up to > the air if it fixes it 100% in all situations as printk() is going to take more > time Well. So. I guess that we don't need irq_work most of the time. We need to queue irq_work for "safe" wake_up_interruptible(), when we know that we can deadlock in scheduler. IOW, only when we are invoked from the scheduler. Scheduler has printk_deferred(), which tells printk() that it cannot do wake_up_interruptible(). Otherwise we can just use normal wake_up_process() and don't need that irq_work->wake_up_interruptible() indirection. The parts of the scheduler, which by mistake call plain printk() from under pi_lock or rq_lock have chances to deadlock anyway and should be switched to printk_deferred(). I think we can queue significantly much less irq_work-s from printk(). Petr, Steven, what do you think? Something like this. Call wake_up_interruptible(), switch to wake_up_klogd() only when called from sched code. --- diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c index cd51aa7d08a9..89cb47882254 100644 --- a/kernel/printk/printk.c +++ b/kernel/printk/printk.c @@ -2027,8 +2027,11 @@ asmlinkage int vprintk_emit(int facility, int level, pending_output = (curr_log_seq != log_next_seq); logbuf_unlock_irqrestore(flags); + if (!pending_output) + return printed_len; + /* If called from the scheduler, we can not call up(). */ - if (!in_sched && pending_output) { + if (!in_sched) { /* * Disable preemption to avoid being preempted while holding * console_sem which would prevent anyone from printing to @@ -2043,10 +2046,11 @@ asmlinkage int vprintk_emit(int facility, int level, if (console_trylock_spinning()) console_unlock(); preempt_enable(); - } - if (pending_output) + wake_up_interruptible(&log_wait); + } else { wake_up_klogd(); + } return printed_len; } EXPORT_SYMBOL(vprintk_emit); --- > and could deal with console hardware that involve irq_exit() anyway. printk->console_driver->write() does not involve irq. > On the other hand, adding __GPF_NOWARN in the build_skb() allocation will fix > this known NET_TX_SOFTIRQ case which is common when softirqd involved at least > in short-term. It even have a benefit to reduce the overall warn_alloc() noise > out there. That's not up to me to decide. -ss