On Tue 2021-11-09 07:27:35, Greg KH wrote: > On Mon, Nov 08, 2021 at 11:17:07AM -0800, Yi Fan wrote: > > On Mon, Nov 8, 2021 at 12:00 AM Greg KH <gregkh@xxxxxxxxxxxxxxxxxxx> wrote: > > > > > > On Thu, Nov 04, 2021 at 12:40:32PM -0700, Yi Fan wrote: > > > > Reply inline. > > > > > > > > On Thu, Nov 4, 2021 at 11:56 AM Greg KH <gregkh@xxxxxxxxxxxxxxxxxxx> wrote: > > > > > > > > > > On Thu, Nov 04, 2021 at 11:14:55AM -0700, Yi Fan wrote: > > > > > > Resend the email using plain text. > > > > > > > > > > > > I found some kernel performance regression issues that might be > > > > > > related w/ 4.14.y LTS commit. > > > > > > > > > > > > 4.14.y commit: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=v4.14.253&id=27d185697322f9547bfd381c71252ce0bc1c0ee4 > > > > > > > > > > > > The issue is observed when "console=" is used as a kernel parameter to > > > > > > disable the kernel console. I think that I see the problem. linux-4.14.y stable branch currently ignores "console=" parameter. As a result, a console (ttyX) is enabled by default. > > > > > What exact "performance issue" are you seeing? > > > > > > > > > [YF] one kernel thread was randomly blocked for more than ~40 > > > > milliseconds, causing a certain task to fail to process in time. > > > > [YF] the issue is highly random on a single device. But it might > > > > happen a few times per 24 hours on a certain percentage of devices. > > > > The overall percentage of devices that show the issue seems quite > > > > stable over a long period of time (somehow the magic number is ~40%.). > > > > [YF] local test on a pool of devices does not show any correlation w/ > > > > any particular devices. This might happen when there is a flood of messages to be printed to the console. It does not happen when there is no console. It has been fixed by the upstream commit 3cffa06aeef7ece30f6b5ac0 ("printk/console: Allow to disable console output by using console="" or console=null") The fix needs some tweaking for the stable branches because __add_preferred_console() has gained more parameters over time. It seems that all longterm stable branches are affected. I am going to prepare the backports. Best Regards, Petr
