System: HP ENVY x360 Convertible 15-ds1xxx; AMD Ryzen 7 4700U with
Radeon Graphics
Problem commits (introduced in v6.9-rc1):
6296562f30b1 HID: amd_sfh: Extend MP2 register access to SFH
2105e8e00da4 HID: amd_sfh: Improve boot time when SFH is available
It appears amd_sfh commits 6296562f30b1 and 2105e8e00da4 correlate with
some form of memory/page corruption. On my system, this typically
presents itself as a page dump followed by BTRFS errors, usually
involving "corrupt leaf" (see dmesg output below); often the BTRFS
filesystem becomes read-only afterwards. Note that the underlying NVME
disk seems fine, and the BTRFS filesystem does not actually appear to be
corrupt when booted/checked from kernels without this bug (no BTRFS
errors or I/O errors reported on non-problem kernels).
I have no problems when I blacklist the amd_sfh module (any kernel
version), or revert both commits 6296562f30b1 and 2105e8e00da4 (on
stable, linux-6.9.y). I have no problems on any recent linux-mainline
(v6.10{,-rc*}) when reverting these two commits (in addition to
reverting 7902ec988a9a and 6856f079cd45 to successfully build the
kernel). I have had no problems with any 6.6.y, v6.7.y, or v6.8.y version.
It is curious BTRFS always seems involved, but problems go away with
these amd_sfh commits reverted (or amd_afh disabled).
Further notes:
I have not specifically used the amd_sfh module for anything. As far
I've been able to determine, my system has the "Sensor Fusion Hub" mp2
chip, but has no supported sensors/sub-devices (or I need to do
something to enable them), (or there is an error while detecting
sensors?). All logs I've checked contain something like:
Jul 09 04:14:37 arch kernel: pcie_mp2_amd 0000:04:00.7: enabling device
(0000 -> 0002)
Jul 09 04:15:07 arch kernel: pcie_mp2_amd 0000:04:00.7: Failed to
discover, sensors not enabled is 0
Jul 09 04:15:07 arch kernel: pcie_mp2_amd 0000:04:00.7:
amd_sfh_hid_client_init failed err -95
Excerpt from lshw:
*-generic:1 UNCLAIMED
description: Signal processing controller
product: Sensor Fusion Hub
vendor: Advanced Micro Devices, Inc. [AMD]
physical id: 0.7
bus info: pci@0000:04:00.7
version: 00
width: 32 bits
clock: 33MHz
capabilities: pm pciexpress msi msix cap_list
configuration: latency=0
resources: memory:fe000000-fe0fffff
memory:fe4cc000-fe4cdfff
How I tracked down the problem commits:
I was not able to successfully "git bisect" this bug - I seemed to run
into a mess of unrelated problems/errors that sent me down a rabbit hole
chasing who knows what. I had already manually narrowed down the bug to
amd_sfh by blacklisting modules, so I reverted each
drivers/hid/amd-sfh-hid commit on the stable linux-6.9.y branch (v6.9.8
known "bad"), back to v6.6 (known "good"), and then manually bisected
the revert commits, landing on "HID: amd_sfh: Improve boot time when SFH
is available" (2105e8e00da4) as the first "bad" commit.
I wanted to be able to test with only the "bad" commit(s) removed; it
turns out 6296562f30b1 ("HID: amd_sfh: Extend MP2 register access to
SFH") needs to be reverted to do that. Everything seems fine with these
two commits reverted (again, this in on the stable linux-6.9.y branch).
When testing, "bad" commits usually quickly display some variation of
the page dump/BTRFS errors, similar to the dmesg output below. I
consider commits "good" if the system survives "stress-ng --all 2
--vm-bytes 50% --minimize --syslog --status 10 -t 5m" (run as a non-root
user), which was usually followed by building the next test kernel. The
"bad" commits often show errors before I even get to the stress test.
Examples of error messages from dmesg:
[ 653.364343] page: refcount:4 mapcount:0 mapping:00000000b159289f
index:0x585a7cec pfn:0x10b5c1
[ 653.364353] memcg:ffff8f2600918000
[ 653.364354] aops:btree_aops ino:1
[ 653.364358] flags:
0x17ffffd000802a(uptodate|lru|private|writeback|node=0|zone=2|lastcpupid=0x1fffff)
[ 653.364361] page_type: 0xffffffff()
[ 653.364363] raw: 0017ffffd000802a fffff1da87ee3288 fffff1da842d70c8
ffff8f260c719458
[ 653.364365] raw: 00000000585a7cec ffff8f26cd09e0f0 00000004ffffffff
ffff8f2600918000
[ 653.364366] page dumped because: eb page dump
[ 653.364367] BTRFS critical (device dm-0): corrupt leaf: root=7
block=6071604133888 slot=159, unexpected item end, have 2768254010
expect 13379
[ 653.364371] BTRFS info (device dm-0): leaf 6071604133888 gen 679995
total ptrs 353 free space 322 owner 7
[ 653.364373] item 0 key (18446744073709551606 128 1062871883776)
itemoff 16271 itemsize 12
[ 653.364375] item 1 key (18446744073709551606 128 1062871896064)
itemoff 16263 itemsize 8
[ 653.364376] item 2 key (18446744073709551606 128 1062871904256)
itemoff 16255 itemsize 8
...
[ 653.364762] item 350 key (18446744073709551606 128 1062879260672)
itemoff 9227 itemsize 12
[ 653.364763] item 351 key (18446744073709551606 128 1062879272960)
itemoff 9223 itemsize 4
[ 653.364764] item 352 key (18446744073709551606 128 1062879277056)
itemoff 9147 itemsize 76
[ 653.364766] BTRFS error (device dm-0): block=6071604133888 write time
tree block corruption detected
[ 653.375440] BTRFS: error (device dm-0) in
btrfs_commit_transaction:2511: errno=-5 IO failure (Error while writing
out transaction)
[ 653.375453] BTRFS info (device dm-0 state E): forced readonly
[ 653.375458] BTRFS warning (device dm-0 state E): Skipping commit of
aborted transaction.
[ 653.375461] BTRFS error (device dm-0 state EA): Transaction aborted
(error -5)
[ 653.375465] BTRFS: error (device dm-0 state EA) in
cleanup_transaction:2005: errno=-5 IO failure
[ 653.375582] BTRFS warning (device dm-0 state EA): Skipping commit of
aborted transaction.
[ 653.375586] BTRFS: error (device dm-0 state EA) in
cleanup_transaction:2005: errno=-5 IO failure
Another example:
[ 5478.134046] page: refcount:4 mapcount:0 mapping:0000000010080c01
index:0x5459ff30 pfn:0x168c7f
[ 5478.134054] memcg:ffff89c240988000
[ 5478.134056] aops:btree_aops ino:1
[ 5478.134061] flags:
0x17ffffd800802a(uptodate|lru|private|writeback|node=0|zone=2|lastcpupid=0x1fffff)
[ 5478.134064] page_type: 0xffffffff()
[ 5478.134066] raw: 0017ffffd800802a ffffcc5d043e2bc8 ffffcc5d05a08c88
ffff89c249968338
[ 5478.134068] raw: 000000005459ff30 ffff89c246fa22d0 00000004ffffffff
ffff89c240988000
[ 5478.134069] page dumped because: eb page dump
[ 5478.134071] BTRFS critical (device dm-0): corrupt leaf: root=2161
block=5796594384896 slot=84 ino=2434728, invalid inode generation: has
72057594122450740 expect (0, 664473]
[ 5478.134075] BTRFS info (device dm-0): leaf 5796594384896 gen 664472
total ptrs 120 free space 1223 owner 2161
[ 5478.134077] item 0 key (2434713 24 3817753667) itemoff 16210 itemsize 73
[ 5478.134078] item 1 key (2434713 108 0) itemoff 15359 itemsize 851
[ 5478.134080] inline extent data size 830
[ 5478.134081] item 2 key (2434714 1 0) itemoff 15199 itemsize 160
[ 5478.134082] inode generation 636724 size 758 mode 100644
[ 5478.134083] item 3 key (2434714 12 2348495) itemoff 15181 itemsize 18
...
[ 5478.134242] item 117 key (2434733 108 0) itemoff 4398 itemsize 329
[ 5478.134243] inline extent data size 308
[ 5478.134244] item 118 key (2434734 1 0) itemoff 4238 itemsize 160
[ 5478.134245] inode generation 636724 size 30 mode 40755
[ 5478.134245] item 119 key (2434734 12 2434375) itemoff 4223 itemsize 15
[ 5478.134247] BTRFS error (device dm-0): block=5796594384896 write time
tree block corruption detected
[ 5478.263726] BTRFS: error (device dm-0) in
btrfs_commit_transaction:2511: errno=-5 IO failure (Error while writing
out transaction)
[ 5478.263733] BTRFS info (device dm-0 state E): forced readonly
[ 5478.263736] BTRFS warning (device dm-0 state E): Skipping commit of
aborted transaction.
[ 5478.263737] BTRFS error (device dm-0 state EA): Transaction aborted
(error -5)
[ 5478.263739] BTRFS: error (device dm-0 state EA) in
cleanup_transaction:2005: errno=-5 IO failure
[ 5478.264582] BTRFS warning (device dm-0 state EA): Skipping commit of
aborted transaction.
[ 5478.264595] BTRFS: error (device dm-0 state EA) in
cleanup_transaction:2005: errno=-5 IO failure
