Hi! Is there anything else I can help here? On Thu, Jun 8, 2023 at 4:03 AM Mikhail Gavrilov <mikhail.v.gavrilov@xxxxxxxxx> wrote: > > Hi, > After beginning the release cycle of the 6.4 kernel I noted that when > I reboot or turn off the computer the last message which I see is a > use-after-free bug found by kasan sanitizer. > Here is photo: https://ibb.co/1fxMYjt > Below photo transcripted to text form: > [ 87.946202] > ================================================================== > [ 87.946247] BUG: KASAN: use-after-free in > tasklet_action_common.isra.0+0x6a4/0x7a0 > [ 87.9462811 Read of size 8 at addr ffff8882b46a6a88 by task ksoftirqd/2/29 > [ 87.946306] > [ 87.946315] CPU: 2 PID: 29 Comm: ksoftirqd/2 Tainted: G W > L ------- --- > 6.4.0-0.rc5.20230606gitf8dba31b0a82.42.fc39.x86_64+debug #1 > [ 87.946359] Hardware name: Micro-Star International Co., Ltd. > MS-7D73/MPG B650I EDGE WIFI (MS-7D73), BIOS 1.30 05/24/2023 > [ 87.946396] Call Trace: > [ 87.946408] <TASK> > [ 87.9464191 dump_stack_lvl+0x76/0xd0 > [ 87.946439] print_report+0xcf/0x670 > [ 87.946459] ? tasklet_action_common.isra.0+0x6a4/0x7a0 > [ 87.946481] ? tasklet action_common.isra.0+0x6a4/0x7a0 > [ 87.946502] kasan_report+0xa8/0xe0 > [ 87.946531] ? tasklet_action_common.isra.0+0x6a4/0x7a0 > [ 87.946555] tasklet_action_common.isra.0+0x6a4/0x7a0 > [ 87.946577] __do_softirq+0x218/0x8bb > [ 87.946596] ? __pfx___do_softirq+0x10/0x10 > [ 87.946614] ? run_ksoftirqd+Ox73/0x80 > [ 87.946633] ? smpboot_thread_fn+0x5bc/0x9b0 > [ 87.946651] run_ksoftirqd+0x4b/0x80 > [ 87.946668] smpboot_thread_fn+0x5bc/0x9b0 > [ 87.946687] ? __pfx_smpboot_thread_fn+0x10/0x10 > [ 87.946706] kthread+0x2eb/0x3c0 > [ 87.946722] ? __pfx_kthread+0x10/0x10 > [ 87.946740] ret_from_fork+0x29/0x50 > [ 87.946760] </TASK> > [ 87.946771] > [ 87.946778] The buggy address belongs to the physical page: > [ 87.946799] page:000000008f30de24 refcount:0 mapcount:0 > mapping:0000000000000000 index:0x0 pfn:0x2b46a6 > [ 87.946833] flags: 0x17ffffc0000000(node=0|zone=2|lastcpupid=0x1fffff) > [ 87.946857] page_type: Oxffffffff() > [ 87.946873] raw: 0017ffffc0000000 0000000000000000 > dead000000000122 0000000000000000 > [ 87.946901] raw: 0000000000000000 0000000000000000 > 00000000ffffffff 0000000000000000 > [ 87.946930] page dumped because: kasan: bad access detected > [ 87.946949] r8169 0000:0e:00.0 enp14s0: Link is Down > [ 87.946950] > [ 87.946968] Memory state around the buggy address: > [ 87.946970] ffff8882b46a6980: ff ff ff ff ff ff ff ff ff ff ff ff > ff ff ff ff > [ 87.946971] ffff8882b46a6a00: ff ff ff ff ff ff ff ff ff ff ff ff > ff ff ff ff > [ 87.946972] >ffff8882b46a6a80: ff ff ff ff ff ff ff ff ff ff ff ff > ff ff ff ff > [ 87.947093] ^ > [ 87.947109] ffff8882b46a6b00: ff ff ff ff ff ff ff ff ff ff ff ff > ff ff ff ff > [ 87.947134] ffff8882b46a6b80: ff ff ff ff ff ff ff ff ff ff ff ff > ff ff ff ff > [ 87.947158] > ================================================================== > [ 87.947186] Disabling lock debugging due to kernel taint > > I suppose many users didn't notice it because all modern Linux distro > use a plymouth screen which hides all kernel messages during boot and > shutdown. And this bug message is not recording in journalctl, because > at the stage when this message appears journalctl was already offline. > > I used git bisect for trying to find the problem commit: > And answered: > - "good" when the computer was finishing work without the > use-after-free message. > - "bad" every time when I saw use-after-free bug message. > - "skip" when the computer was stucking at the shutdown. > And I got such bisect log: > > git bisect start > # status: waiting for both good and bad commits > # good: [173ea743bf7a9eef04460e03b00ba267cc52aee2] Merge tag > 'pull-nios2' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs > git bisect good 173ea743bf7a9eef04460e03b00ba267cc52aee2 > # status: waiting for bad commit, 1 good commit known > # bad: [6e98b09da931a00bf4e0477d0fa52748bf28fcce] Merge tag > 'net-next-6.4' of > git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next > git bisect bad 6e98b09da931a00bf4e0477d0fa52748bf28fcce > # good: [2c96606a0f8b7900387dbeb6532b59527999834d] Merge tag > 'gpio-updates-for-v6.4' of > git://git.kernel.org/pub/scm/linux/kernel/git/brgl/linux > git bisect good 2c96606a0f8b7900387dbeb6532b59527999834d > # bad: [ca288965801572fe41386560d4e6c5cc0e5cc56d] Merge tag > 'wireless-next-2023-04-21' of > git://git.kernel.org/pub/scm/linux/kernel/git/wireless/wireless-next > git bisect bad ca288965801572fe41386560d4e6c5cc0e5cc56d > # good: [d56417ad1133fc41752bb9fe37da7ae3187395a4] net: phy: smsc: > clear edpd_enable if interrupt mode is used > git bisect good d56417ad1133fc41752bb9fe37da7ae3187395a4 > # good: [c54876cd5961ce0f8e74807f79a6739cd6b35ddf] net/sched: pass > netlink extack to mqprio and taprio offload > git bisect good c54876cd5961ce0f8e74807f79a6739cd6b35ddf > # skip: [3288ee5844b74cebb94ed15bc9b5b9d3223ae038] Merge ath-next from > git://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git > git bisect skip 3288ee5844b74cebb94ed15bc9b5b9d3223ae038 > # good: [b6d85cf5bd1433c5dd6bf6bb3a176537184c630c] net/ipv6: > Initialise msg_control_is_user > git bisect good b6d85cf5bd1433c5dd6bf6bb3a176537184c630c > # skip: [d2a158d113cbfe37a5dd3f44dc96d008dd910a81] Merge tag > 'mt76-for-kvalo-2023-04-18' of https://github.com/nbd168/wireless > git bisect skip d2a158d113cbfe37a5dd3f44dc96d008dd910a81 > # good: [02461d9368c59510ef51cc8a1db1f0f31cfbf9ad] wifi: rtw88: main: > Reserve 8 bytes of extra TX headroom for SDIO cards > git bisect good 02461d9368c59510ef51cc8a1db1f0f31cfbf9ad > # good: [827145392a4aad635b93e5235b7d7fecc2fa31c7] net: enetc: only > commit preemptible TCs to hardware when MM TX is active > git bisect good 827145392a4aad635b93e5235b7d7fecc2fa31c7 > # skip: [27db47ab1f47906c2392f9d246e244e412b19278] wifi: mt76: mt7996: > enable mesh HW amsdu/de-amsdu support > git bisect skip 27db47ab1f47906c2392f9d246e244e412b19278 > # good: [22b68fc6d693e7a2b1c0eb852463f4a72522fa08] wifi: iwlwifi: mvm: > fix RFKILL report when driver is going down > git bisect good 22b68fc6d693e7a2b1c0eb852463f4a72522fa08 > # good: [f94557154d9fc77c392844523388edd4661a27a3] wifi: wcn36xx: add > support for pronto-v3 > git bisect good f94557154d9fc77c392844523388edd4661a27a3 > # good: [ccf73f6e69c0244a979e97eb6c38f80cd6cbc116] wifi: rtw88: add > port switch for AP mode > git bisect good ccf73f6e69c0244a979e97eb6c38f80cd6cbc116 > # good: [a6f187f92bcc2b17821538b4a11d61764e68b091] wifi: rtw88: usb: > fix priority queue to endpoint mapping > git bisect good a6f187f92bcc2b17821538b4a11d61764e68b091 > # skip: [61d1f54533496711e06fcfd42b93c5ded9e27c7a] wifi: mt76: move > mcu_uni_event and mcu_reg_event in common code > git bisect skip 61d1f54533496711e06fcfd42b93c5ded9e27c7a > # good: [73175a042955e531ec355a8708585befa67a22db] sctp: delete the > nested flexible array skip > git bisect good 73175a042955e531ec355a8708585befa67a22db > # good: [b9235aef84929e5330cb87125a6baf1cf7988983] wifi: ath12k: > Remove redundant pci_clear_master > git bisect good b9235aef84929e5330cb87125a6baf1cf7988983 > # good: [6257c702264c44d74c6b71f0c62a7665da2dc356] wifi: ath11k: fix > tx status reporting in encap offload mode > git bisect good 6257c702264c44d74c6b71f0c62a7665da2dc356 > # skip: [3b522cadedfe6e9e0e8193d7d4ab5aa8d0c73209] wifi: mt76: mt7996: > fill txd by host driver > git bisect skip 3b522cadedfe6e9e0e8193d7d4ab5aa8d0c73209 > # skip: [f4d63a87b527de258eec5bd6e9547f063d472b79] wifi: mt76: dma: > use napi_build_skb > git bisect skip f4d63a87b527de258eec5bd6e9547f063d472b79 > # skip: [09d4d6da1b65d09414e7bce61459593f3c80ead1] wifi: mt76: > mt7921e: Set memory space enable in PCI_COMMAND if unset > git bisect skip 09d4d6da1b65d09414e7bce61459593f3c80ead1 > # skip: [230a167e094770fdcc104481719ef7b1a706fb27] wifi: mt76: set > NL80211_EXT_FEATURE_CAN_REPLACE_PTK0 on supported drivers > git bisect skip 230a167e094770fdcc104481719ef7b1a706fb27 > # good: [49ce92fbee0b6bb8066dddf37489483b3b6b5c25] pds_core: add FW > update feature to devlink > git bisect good 49ce92fbee0b6bb8066dddf37489483b3b6b5c25 > # skip: [12db28c3ef31f719bd18fa186a40bb152e6a527c] mt76: mt7921: fix > kernel panic by accessing unallocated eeprom.data > git bisect skip 12db28c3ef31f719bd18fa186a40bb152e6a527c > # good: [45fd01f2fbf1119d083931b095ad6d0f13443d0e] net/mlx5e: Refactor > duplicated code in mlx5e_ipsec_init_macs > git bisect good 45fd01f2fbf1119d083931b095ad6d0f13443d0e > # skip: [2631c5b6ef9d7c6707e020def6947464c8aa6f92] wifi: mt76: Replace > zero-length array with flexible-array member > git bisect skip 2631c5b6ef9d7c6707e020def6947464c8aa6f92 > # good: [64822bdba456a145f7cb4c66d9939bf42c64ae62] dt-bindings: mt76: > add active-low property for led > git bisect good 64822bdba456a145f7cb4c66d9939bf42c64ae62 > # skip: [6d6793cef6a491b8f6db5f40ef3334411293da32] wifi: mt76: mt7921: > Replace fake flex-arrays with flexible-array members > git bisect skip 6d6793cef6a491b8f6db5f40ef3334411293da32 > # skip: [3d78c46423c6567ed25ca033e086865b1b4d5ae1] wifi: mt76: > mt7921e: stop chip reset worker in unregister hook > git bisect skip 3d78c46423c6567ed25ca033e086865b1b4d5ae1 > # good: [b100722a777f6455d913666a376f81342b2cb995] wifi: ath11k: > Remove disabling of 80+80 and 160 MHz > git bisect good b100722a777f6455d913666a376f81342b2cb995 > # skip: [03eb52dd78cab08f13925aeec8315fbdbcba3253] wifi: mt76: mt7921: > add Netgear AXE3000 (A8000) support > git bisect skip 03eb52dd78cab08f13925aeec8315fbdbcba3253 > # good: [6a8b899df1562a46a8c55cebc7d35508a24300d3] wifi: mt76: add > mt76_connac_gen_ppe_thresh utility routine > git bisect good 6a8b899df1562a46a8c55cebc7d35508a24300d3 > # skip: [15ee62e73705df447971613de4fa660dd71ed40e] wifi: mt76: mt7996: > enable BSS_CHANGED_BASIC_RATES support > git bisect skip 15ee62e73705df447971613de4fa660dd71ed40e > # skip: [5c47cdebbaeb7724df6f9f46917c93e53f791547] wifi: mt76: mt7921: > fix missing unwind goto in `mt7921u_probe` > git bisect skip 5c47cdebbaeb7724df6f9f46917c93e53f791547 > # good: [97c75e1adeda78b3794936c617d8b86e9ebd54f5] wifi: rtw88: set > pkg_type correctly for specific rtw8821c variants > git bisect good 97c75e1adeda78b3794936c617d8b86e9ebd54f5 > # good: [59a3a312009723e3e5082899655fdcc420e2b47a] wifi: rtw88: Fix > memory leak in rtw88_usb > git bisect good 59a3a312009723e3e5082899655fdcc420e2b47a > # skip: [3d2892e05086d09aecf14ea64b2debbf495e313c] wifi: mt76: connac: > fix txd multicast rate setting > git bisect skip 3d2892e05086d09aecf14ea64b2debbf495e313c > # good: [c2171b068beea766311e4c2858ef8497504c6e6d] wifi: mt76: mt7996: > enable configured beacon tx rate > git bisect good c2171b068beea766311e4c2858ef8497504c6e6d > # good: [cd85c8b059c54b00e3b509e83fb36c2798f50128] wifi: rtl8xxxu: Add > rtl8xxxu_write{8,16,32}_{set,clear} > git bisect good cd85c8b059c54b00e3b509e83fb36c2798f50128 > # skip: [e12b2e99b8799f26432528934edc8677888ad72f] wifi: mt76: mt7615: > increase eeprom size for mt7663 > git bisect skip e12b2e99b8799f26432528934edc8677888ad72f > # good: [6c6d62ae8271bd4b55dd2ba4b7ed552162823880] wifi: rtw88: Update > spelling in main.h > git bisect good 6c6d62ae8271bd4b55dd2ba4b7ed552162823880 > # good: [dfc39d4026fb2432363c0f77543c4cf3adca4c7b] net/packet: support > mergeable feature of virtio > git bisect good dfc39d4026fb2432363c0f77543c4cf3adca4c7b > # only skipped commits left to test > # possible first bad commit: > [ca288965801572fe41386560d4e6c5cc0e5cc56d] Merge tag > 'wireless-next-2023-04-21' of > git://git.kernel.org/pub/scm/linux/kernel/git/wireless/wireless-next > # possible first bad commit: > [3288ee5844b74cebb94ed15bc9b5b9d3223ae038] Merge ath-next from > git://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git > # possible first bad commit: > [d2a158d113cbfe37a5dd3f44dc96d008dd910a81] Merge tag > 'mt76-for-kvalo-2023-04-18' of https://github.com/nbd168/wireless > # possible first bad commit: > [3b522cadedfe6e9e0e8193d7d4ab5aa8d0c73209] wifi: mt76: mt7996: fill > txd by host driver > # possible first bad commit: > [230a167e094770fdcc104481719ef7b1a706fb27] wifi: mt76: set > NL80211_EXT_FEATURE_CAN_REPLACE_PTK0 on supported drivers > # possible first bad commit: > [f4d63a87b527de258eec5bd6e9547f063d472b79] wifi: mt76: dma: use > napi_build_skb > # possible first bad commit: > [e12b2e99b8799f26432528934edc8677888ad72f] wifi: mt76: mt7615: > increase eeprom size for mt7663 > # possible first bad commit: > [27db47ab1f47906c2392f9d246e244e412b19278] wifi: mt76: mt7996: enable > mesh HW amsdu/de-amsdu support > > Unfortunately git bisect did not say which exact commit is culprit > here, but anyway we got the result with eigh commits which can be > reviewed. -- Best Regards, Mike Gavrilov.
