On Thu, Jun 8, 2023 at 7:12 AM Johan Hovold <johan@xxxxxxxxxx> wrote: > > Hi Rob, > > Have you had a chance to look at this regression yet? It prevents us > from using lockdep on the X13s as it is disabled as soon as we start > the GPU. Hmm, curious what is different between x13s and sc7180/sc7280 things? Or did lockdep recently get more clever (or more annotation)? I did spend some time a while back trying to bring some sense to devfreq/pm-qos/icc locking: https://patchwork.freedesktop.org/series/115028/ but haven't had time to revisit that for a while BR, -R > On Wed, Mar 15, 2023 at 10:19:21AM +0100, Johan Hovold wrote: > > > > Since 6.3-rc2 (or possibly -rc1), I'm now seeing the below > > devfreq-related lockdep splat. > > > > I noticed that you posted a fix for something similar here: > > > > https://lore.kernel.org/r/20230312204150.1353517-9-robdclark@xxxxxxxxx > > > > but that particular patch makes no difference. > > > > From skimming the calltraces below and qos/devfreq related changes in > > 6.3-rc1 it seems like this could be related to: > > > > fadcc3ab1302 ("drm/msm/gpu: Bypass PM QoS constraint for idle clamp") > > Below is an updated splat from 6.4-rc5. > > Johan > > [ 2941.931507] ====================================================== > [ 2941.931509] WARNING: possible circular locking dependency detected > [ 2941.931513] 6.4.0-rc5 #64 Not tainted > [ 2941.931516] ------------------------------------------------------ > [ 2941.931518] ring0/359 is trying to acquire lock: > [ 2941.931520] ffff63310e35c078 (&devfreq->lock){+.+.}-{3:3}, at: qos_min_notifier_call+0x28/0x88 > [ 2941.931541] > but task is already holding lock: > [ 2941.931543] ffff63310e3cace8 (&(c->notifiers)->rwsem){++++}-{3:3}, at: blocking_notifier_call_chain+0x30/0x70 > [ 2941.931553] > which lock already depends on the new lock. > > [ 2941.931555] > the existing dependency chain (in reverse order) is: > [ 2941.931556] > -> #4 (&(c->notifiers)->rwsem){++++}-{3:3}: > [ 2941.931562] down_write+0x50/0x198 > [ 2941.931567] blocking_notifier_chain_register+0x30/0x8c > [ 2941.931570] freq_qos_add_notifier+0x68/0x7c > [ 2941.931574] dev_pm_qos_add_notifier+0xa0/0xf8 > [ 2941.931579] devfreq_add_device.part.0+0x360/0x5a4 > [ 2941.931583] devm_devfreq_add_device+0x74/0xe0 > [ 2941.931587] msm_devfreq_init+0xa0/0x154 [msm] > [ 2941.931624] msm_gpu_init+0x2fc/0x588 [msm] > [ 2941.931649] adreno_gpu_init+0x160/0x2d0 [msm] > [ 2941.931675] a6xx_gpu_init+0x2c0/0x74c [msm] > [ 2941.931699] adreno_bind+0x180/0x290 [msm] > [ 2941.931723] component_bind_all+0x124/0x288 > [ 2941.931728] msm_drm_bind+0x1d8/0x6cc [msm] > [ 2941.931752] try_to_bring_up_aggregate_device+0x1ec/0x2f4 > [ 2941.931755] __component_add+0xa8/0x194 > [ 2941.931758] component_add+0x14/0x20 > [ 2941.931761] dp_display_probe+0x2b4/0x474 [msm] > [ 2941.931785] platform_probe+0x68/0xd8 > [ 2941.931789] really_probe+0x184/0x3c8 > [ 2941.931792] __driver_probe_device+0x7c/0x16c > [ 2941.931794] driver_probe_device+0x3c/0x110 > [ 2941.931797] __device_attach_driver+0xbc/0x158 > [ 2941.931800] bus_for_each_drv+0x84/0xe0 > [ 2941.931802] __device_attach+0xa8/0x1d4 > [ 2941.931805] device_initial_probe+0x14/0x20 > [ 2941.931807] bus_probe_device+0xb0/0xb4 > [ 2941.931810] deferred_probe_work_func+0xa0/0xf4 > [ 2941.931812] process_one_work+0x288/0x5bc > [ 2941.931816] worker_thread+0x74/0x450 > [ 2941.931818] kthread+0x124/0x128 > [ 2941.931822] ret_from_fork+0x10/0x20 > [ 2941.931826] > -> #3 (dev_pm_qos_mtx){+.+.}-{3:3}: > [ 2941.931831] __mutex_lock+0xa0/0x840 > [ 2941.931833] mutex_lock_nested+0x24/0x30 > [ 2941.931836] dev_pm_qos_remove_notifier+0x34/0x140 > [ 2941.931838] genpd_remove_device+0x3c/0x174 > [ 2941.931841] genpd_dev_pm_detach+0x78/0x1b4 > [ 2941.931844] dev_pm_domain_detach+0x24/0x34 > [ 2941.931846] a6xx_gmu_remove+0x34/0xc4 [msm] > [ 2941.931869] a6xx_destroy+0xd0/0x160 [msm] > [ 2941.931892] adreno_unbind+0x40/0x64 [msm] > [ 2941.931916] component_unbind+0x38/0x6c > [ 2941.931919] component_unbind_all+0xc8/0xd4 > [ 2941.931921] msm_drm_uninit.isra.0+0x150/0x1c4 [msm] > [ 2941.931945] msm_drm_bind+0x310/0x6cc [msm] > [ 2941.931967] try_to_bring_up_aggregate_device+0x1ec/0x2f4 > [ 2941.931970] __component_add+0xa8/0x194 > [ 2941.931973] component_add+0x14/0x20 > [ 2941.931976] dp_display_probe+0x2b4/0x474 [msm] > [ 2941.932000] platform_probe+0x68/0xd8 > [ 2941.932003] really_probe+0x184/0x3c8 > [ 2941.932005] __driver_probe_device+0x7c/0x16c > [ 2941.932008] driver_probe_device+0x3c/0x110 > [ 2941.932011] __device_attach_driver+0xbc/0x158 > [ 2941.932014] bus_for_each_drv+0x84/0xe0 > [ 2941.932016] __device_attach+0xa8/0x1d4 > [ 2941.932018] device_initial_probe+0x14/0x20 > [ 2941.932021] bus_probe_device+0xb0/0xb4 > [ 2941.932023] deferred_probe_work_func+0xa0/0xf4 > [ 2941.932026] process_one_work+0x288/0x5bc > [ 2941.932028] worker_thread+0x74/0x450 > [ 2941.932031] kthread+0x124/0x128 > [ 2941.932035] ret_from_fork+0x10/0x20 > [ 2941.932037] > -> #2 (&gmu->lock){+.+.}-{3:3}: > [ 2941.932043] __mutex_lock+0xa0/0x840 > [ 2941.932045] mutex_lock_nested+0x24/0x30 > [ 2941.932047] a6xx_gpu_set_freq+0x30/0x5c [msm] > [ 2941.932071] msm_devfreq_target+0xb8/0x1a8 [msm] > [ 2941.932094] devfreq_set_target+0x84/0x27c > [ 2941.932098] devfreq_update_target+0xc4/0xec > [ 2941.932102] devfreq_monitor+0x38/0x170 > [ 2941.932105] process_one_work+0x288/0x5bc > [ 2941.932108] worker_thread+0x74/0x450 > [ 2941.932110] kthread+0x124/0x128 > [ 2941.932113] ret_from_fork+0x10/0x20 > [ 2941.932116] > -> #1 (&df->lock){+.+.}-{3:3}: > [ 2941.932121] __mutex_lock+0xa0/0x840 > [ 2941.932124] mutex_lock_nested+0x24/0x30 > [ 2941.932126] msm_devfreq_get_dev_status+0x48/0x134 [msm] > [ 2941.932149] devfreq_simple_ondemand_func+0x3c/0x144 > [ 2941.932153] devfreq_update_target+0x4c/0xec > [ 2941.932157] devfreq_monitor+0x38/0x170 > [ 2941.932160] process_one_work+0x288/0x5bc > [ 2941.932162] worker_thread+0x74/0x450 > [ 2941.932165] kthread+0x124/0x128 > [ 2941.932168] ret_from_fork+0x10/0x20 > [ 2941.932171] > -> #0 (&devfreq->lock){+.+.}-{3:3}: > [ 2941.932175] __lock_acquire+0x13d8/0x2188 > [ 2941.932178] lock_acquire+0x1e8/0x310 > [ 2941.932180] __mutex_lock+0xa0/0x840 > [ 2941.932182] mutex_lock_nested+0x24/0x30 > [ 2941.932184] qos_min_notifier_call+0x28/0x88 > [ 2941.932188] notifier_call_chain+0xa0/0x17c > [ 2941.932190] blocking_notifier_call_chain+0x48/0x70 > [ 2941.932193] pm_qos_update_target+0xdc/0x1d0 > [ 2941.932195] freq_qos_apply+0x68/0x74 > [ 2941.932198] apply_constraint+0x100/0x148 > [ 2941.932201] __dev_pm_qos_update_request+0xb8/0x1fc > [ 2941.932203] dev_pm_qos_update_request+0x3c/0x64 > [ 2941.932206] msm_devfreq_active+0xf8/0x194 [msm] > [ 2941.932227] msm_gpu_submit+0x18c/0x1a8 [msm] > [ 2941.932249] msm_job_run+0x98/0x11c [msm] > [ 2941.932272] drm_sched_main+0x1a0/0x444 [gpu_sched] > [ 2941.932281] kthread+0x124/0x128 > [ 2941.932284] ret_from_fork+0x10/0x20 > [ 2941.932287] > other info that might help us debug this: > > [ 2941.932289] Chain exists of: > &devfreq->lock --> dev_pm_qos_mtx --> &(c->notifiers)->rwsem > > [ 2941.932296] Possible unsafe locking scenario: > > [ 2941.932298] CPU0 CPU1 > [ 2941.932300] ---- ---- > [ 2941.932301] rlock(&(c->notifiers)->rwsem); > [ 2941.932304] lock(dev_pm_qos_mtx); > [ 2941.932307] lock(&(c->notifiers)->rwsem); > [ 2941.932309] lock(&devfreq->lock); > [ 2941.932312] > *** DEADLOCK *** > > [ 2941.932313] 4 locks held by ring0/359: > [ 2941.932315] #0: ffff633110966170 (&gpu->lock){+.+.}-{3:3}, at: msm_job_run+0x8c/0x11c [msm] > [ 2941.932342] #1: ffff633110966208 (&gpu->active_lock){+.+.}-{3:3}, at: msm_gpu_submit+0xdc/0x1a8 [msm] > [ 2941.932368] #2: ffffa40da2f91ed0 (dev_pm_qos_mtx){+.+.}-{3:3}, at: dev_pm_qos_update_request+0x30/0x64 > [ 2941.932374] #3: ffff63310e3cace8 (&(c->notifiers)->rwsem){++++}-{3:3}, at: blocking_notifier_call_chain+0x30/0x70 > [ 2941.932381] > stack backtrace: > [ 2941.932383] CPU: 7 PID: 359 Comm: ring0 Not tainted 6.4.0-rc5 #64 > [ 2941.932386] Hardware name: LENOVO 21BYZ9SRUS/21BYZ9SRUS, BIOS N3HET53W (1.25 ) 10/12/2022 > [ 2941.932389] Call trace: > [ 2941.932391] dump_backtrace+0x9c/0x11c > [ 2941.932395] show_stack+0x18/0x24 > [ 2941.932398] dump_stack_lvl+0x60/0xac > [ 2941.932402] dump_stack+0x18/0x24 > [ 2941.932405] print_circular_bug+0x26c/0x348 > [ 2941.932407] check_noncircular+0x134/0x148 > [ 2941.932409] __lock_acquire+0x13d8/0x2188 > [ 2941.932411] lock_acquire+0x1e8/0x310 > [ 2941.932414] __mutex_lock+0xa0/0x840 > [ 2941.932416] mutex_lock_nested+0x24/0x30 > [ 2941.932418] qos_min_notifier_call+0x28/0x88 > [ 2941.932421] notifier_call_chain+0xa0/0x17c > [ 2941.932424] blocking_notifier_call_chain+0x48/0x70 > [ 2941.932426] pm_qos_update_target+0xdc/0x1d0 > [ 2941.932428] freq_qos_apply+0x68/0x74 > [ 2941.932431] apply_constraint+0x100/0x148 > [ 2941.932433] __dev_pm_qos_update_request+0xb8/0x1fc > [ 2941.932435] dev_pm_qos_update_request+0x3c/0x64 > [ 2941.932437] msm_devfreq_active+0xf8/0x194 [msm] > [ 2941.932460] msm_gpu_submit+0x18c/0x1a8 [msm] > [ 2941.932482] msm_job_run+0x98/0x11c [msm] > [ 2941.932504] drm_sched_main+0x1a0/0x444 [gpu_sched] > [ 2941.932511] kthread+0x124/0x128 > [ 2941.932514] ret_from_fork+0x10/0x20
