Re: [OSADL QA 3.18.9-rt4 #1] Radeon driver hangs

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Michel,

[..]
The most striking problem of kernel 3.18.9-rt4 affects all systems that
are equipped with Radeon graphics (irrespective whether PCIe cards or
APUs with on-chip graphics). They suffer from a hanging radeon driver.
The block occurs when accelerated graphics load is created by x11perf or
gltestperf. Sometimes only the graphics are frozen while ssh login still
is possible, somtimes the entire box is no longer accessible at all. In
any case, a reboot is needed to recover from this situation.

Here is a selection of kernel messages:
[...]
The commits from
http://cgit.freedesktop.org/~airlied/linux/commit/?h=drm-fixes&id=f957063fee6392bb9365370db6db74dc0b2dce0a

to
http://cgit.freedesktop.org/~airlied/linux/commit/?h=drm-fixes&id=cffefd9bb31cd35ab745d3b49005d10616d25bdc

and
http://cgit.freedesktop.org/~airlied/linux/commit/?h=drm-fixes&id=b6610101718d4ab90d793c482625e98eb1262cad

might help for this.

Thanks a lot. I have applied these patches to a number of systems:
# quilt applied | tail -7
patches/drm-radeon-do-a-posting-read-in-r100_set_irq.patch
patches/drm-radeon-do-a-posting-read-in-rs600_set_irq.patch
patches/drm-radeon-do-a-posting-read-in-r600_set_irq.patch
patches/drm-radeon-do-a-posting-read-in-evergreen_set_irq.patch
patches/drm-radeon-do-a-posting-read-in-si_set_irq.patch
patches/drm-radeon-do-a-posting-read-in-cik_set_irq.patch
patches/drm-radeon-fix-wait-to-actually-occur-after-the-signaling-callback.patch


  The graphic boards still crash and freeze the screen, but in contrast
to the earlier situation the systems remain accessible, and the X
Window server can be restarted after the offensive programs are
removed. The crashes were reliably triggered by
- gltestperf
   or
- x11perf -repeat 3 -subs 25 -time 2 -rect10
This is not entirely correct, since gltestperf does not reliably crash
the graphics controller. However, "x11perf -repeat 3 -subs 25 -time 2
-rect10" always does a reliable job to trigger the crash.

but the crashes also occur several times per day during normal work
such as browsing the Internet or writing a text document. If you wish
me to provide additional diagnostic information such as running test
programs while the graphic boards are unresponsive, I certainly can do
that.

Does it also happen with a kernel built from a current drm-fixes tree?
http://cgit.freedesktop.org/~airlied/linux/log/?h=drm-fixes
No. Apparently, you need full preemption to expose the problem.

The following list contains the results whether the command "x11perf
-repeat 3 -subs 25 -time 2 -rect10" freezes the Radeon board under test
(Radeon HD 7970 XFS / R9 280X) or not:
linux-3.12.33-rt47               no
linux-3.14.34-rt32               no
linux-3.14.34-drm-3.16.7-rt32*   no
linux-3.18.7-rt1                YES
linux-3.18.9-rt4                YES
linux-3.18.9-rt5                YES
linux-3.18.9-drm-3.16.7-rt5**    no
linux-4.0.0-rc4                  no
linux-drm-fixes                  no
*DRM subsystem backported from linux-3.16.7 to linux-3.14.34-rt32.
**DRM subsystem ported from linux-3.16.7 to linux-3.18.9-rt5.

More observations:
If full function tracing is enabled (which makes the system about five
times slower), the graphics controller no longer freezes. With partial
function tracing such as "echo *drm* >set_ftrace_filter", the
controller still freezes. The trace then contains vblank interrupt
processing only, ioctls are no longer executed.

This is the location where the driver hangs:
[25104.509258] INFO: task Xorg.bin:16591 blocked for more than 120 seconds.
[25104.516322]       Not tainted 3.18.9-rt5 #2
[25104.520715] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [25104.528853] Xorg.bin D ffffffff8171ed90 0 16591 16239 0x10400080 [25104.536102] ffff8800ba0bb8d8 0000000000000002 ffff8800ba0bbfd8 0000000000000006 [25104.536103] 000000000000dc08 ffff880626d0dc08 ffff8800ba0bbfd8 000000000000dc08 [25104.536104] ffff88061b2cdcd0 ffff880616d3a940 ffff880035c10000 ffff880616d3a940
[25104.559274] Call Trace:
[25104.561844]  [<ffffffff8171bb54>] schedule+0x34/0xa0
[25104.561846]  [<ffffffff8171e2ac>] schedule_timeout+0x23c/0x2a0
[25104.561870] [<ffffffffa00e3ab6>] ? radeon_fence_process+0x16/0x40 [radeon] [25104.561879] [<ffffffffa00e3b24>] ? radeon_fence_any_seq_signaled+0x44/0x90 [radeon] [25104.561887] [<ffffffffa00e3e97>] radeon_fence_wait_seq_timeout.constprop.8+0x327/0x380 [radeon]
[25104.561889]  [<ffffffff810d19c0>] ? __wake_up_sync+0x20/0x20
[25104.561898] [<ffffffffa00e4287>] radeon_fence_wait_any+0x57/0x70 [radeon]
[25104.561914]  [<ffffffffa015a36f>] radeon_sa_bo_new+0x2af/0x4b0 [radeon]
[25104.561916]  [<ffffffff81379b07>] ? debug_smp_processor_id+0x17/0x20
[25104.561918]  [<ffffffff811d0b4a>] ? __kmalloc+0x8a/0x300
[25104.561932]  [<ffffffffa01b2197>] radeon_ib_get+0x37/0xe0 [radeon]
[25104.561943]  [<ffffffffa01003ee>] radeon_cs_ioctl+0x22e/0x860 [radeon]
[25104.561952]  [<ffffffffa0005bc7>] drm_ioctl+0x197/0x670 [drm]
[25104.561954]  [<ffffffff81379b07>] ? debug_smp_processor_id+0x17/0x20
[25104.561956]  [<ffffffff810901ba>] ? unpin_current_cpu+0x1a/0x80
[25104.561959]  [<ffffffff810ba200>] ? migrate_enable+0x90/0x1a0
[25104.561966]  [<ffffffffa00c604c>] radeon_drm_ioctl+0x4c/0x80 [radeon]
[25104.561967]  [<ffffffff811fdb88>] do_vfs_ioctl+0x2c8/0x4c0
[25104.561969]  [<ffffffff81208a92>] ? __fget+0x72/0xb0
[25104.561970]  [<ffffffff811fde01>] SyS_ioctl+0x81/0xa0
[25104.561971]  [<ffffffff8171f99e>] tracesys_phase2+0xd4/0xd9

Conclusion:
An upgrade change of the DRM subsystem between 3.16.7 and 3.18.9
introduced a race condition that freezes Radeon graphics. It requires
full preemption to be exposed reliably.

Thanks,
	-Carsten.
_______________________________________________
dri-devel mailing list
dri-devel@xxxxxxxxxxxxxxxxxxxxx
http://lists.freedesktop.org/mailman/listinfo/dri-devel





[Index of Archives]     [Linux DRI Users]     [Linux Intel Graphics]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [XFree86]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Linux Kernel]     [Linux SCSI]     [XFree86]
  Powered by Linux