Hi there, I appear to have stumbled upon a deadlock in wbt or rq-qos. My journal of a lot of data points is over here: https://github.com/openzfs/zfs/issues/12204 I initially deadlocked on RHEL 8.4's 4.18.0-305.3.1.el8_4.x86_64 kernel, but the code in blk-wbt.c / blk-rq-qos.c is functionally identical to 5.13.0-rc5, so I tried that and I'm able to deadlock that as well. I believe the same code exists all the way back to 5.0.1. The Something Weird (tm) about this is that it possibly only happens on AMD EPYC CPUs. I just don't have the necessary setup to confirm that either way, but it's a hunch because I can't reproduce it on an Ice Lake VM (but the Ice Lake VM also has more storage bandwidth so that could be the thing, and I can't decrease that storage bandwidth, so I can't do a like-for-like test.) I "instrumented" wbt / rq-qos with a bunch of printk's which you can see with this patch: https://gist.github.com/omarkilani/2ad526c3546b40537b546450c8f685dc I then ran my repro workload to cause the deadlock, here's the dmesg output just before the deadlock and then the backtraces with my printk patch applied: https://gist.githubusercontent.com/omarkilani/ff0a96d872e09b4fb648272d104e0053/raw/d3da3974162f8aa87b7309317af80929fadf250f/dmesg.wbt.deadlock.log Happy to apply whatever / run whatever to get more data. Thanks! Regards, Omar
