Hi, this is your Linux kernel regression tracker. Top-posting for once, to make this easily accessible to everyone. FYI: looks like this isn't a regression, as Stephane mentioned in a comment to the bko report: https://bugzilla.kernel.org/show_bug.cgi?id=215660#c13 > So today i could reproduce on 5.10.46 after a few days of testing. It does not look like a regression. Thus removing it from the regression tracking: #regzbot invalid: After further testing it does not look like a regression anymore Ciao, Thorsten (wearing his 'the Linux kernel's regression tracker' hat) P.S.: As the Linux kernel's regression tracker I'm getting a lot of reports on my table. I can only look briefly into most of them and lack knowledge about most of the areas they concern. I thus unfortunately will sometimes get things wrong or miss something important. I hope that's not the case here; if you think it is, don't hesitate to tell me in a public reply, it's in everyone's interest to set the public record straight. On 14.03.22 12:44, Thorsten Leemhuis wrote: > Hi, this is your Linux kernel regression tracker. > > I noticed a regression report in bugzilla.kernel.org that afaics nobody > acted upon since it was reported more than ten days ago (it afaifcs only > later became clear this is a regression), that's why I decided to > forward it to the lists and a few relevant people to the CC. To quote > from https://bugzilla.kernel.org/show_bug.cgi?id=215660: > >> Stephane Poignant 2022-03-04 17:24:49 UTC >> >> Created attachment 300529 [details] >> lspci and ethtool outputs on reproducing systems >> >> Context: >> - dense enterprise deployment, 10 lightweight aps (Aruba) on one office floor, up to 125 concurrent users total, up to 25 user per AP >> - the wireless network supports 802.11n, 802.11ac and 802.11ax in 5 GHz band >> - authentication is wpa2-psk >> - client devices consists in a variety of endpoints (laptops, cell phones, tablets, smart devices), running various versions of Mac OSX, Linux, Windows, Android or IOS. >> - certain clients supports only 20Mhz, HT protection kicks in and turns off on APs as those clients are moving around. Consequently ht_operation_mode fluctuates between 4 and 6 even when staying on the same AP. >> - the issue affects various laptops with Intel AX200 or AX201 chipsets, running Debian or Ubuntu with a recent kernel >= 5.10 >> - see attached file devices.txt for detailed information on the different laptops we have reproduced the issue on >> >> >> Steps to reproduce: >> - appears sometimes, but not always, after the iwlwifi STA roams from one AP to another >> - seen more often when ht_operation_mode changes between 4 and 6 (but not sufficient to trigger the issue) >> - STA deassociates from current AP and associates to the new one successfully >> - connectivity works on the new AP for a short period of time, usually between 30s and 1 minute >> - then suddenly, the Rx path breaks. No more received frame visible on the STA wireless interface. AP reports that frames are retransmitted and not acknowledged by STA. >> - the Tx path keeps working. Frames sent by STA to AP are received and visible on the network >> - in this state each inbound frame appears to trigger iwl_pcie_rx_handle_rb with cmd BAR_FRAME_RELEASE (seqnum is always the same): >> >> Mar 4 12:44:32 debian kernel: [15884.715812] iwlwifi 0000:00:14.3: iwl_pcie_rx_handle Q 0: HW = 338, SW = 337 >> Mar 4 12:44:32 debian kernel: [15884.715819] iwlwifi 0000:00:14.3: iwl_pcie_get_rxb Got virtual RB ID 1348 >> Mar 4 12:44:32 debian kernel: [15884.715831] iwlwifi 0000:00:14.3: iwl_pcie_rx_handle_rb Q 0: cmd at offset 0: BAR_FRAME_RELEASE (00.c2, seq 0xbfff) >> Mar 4 12:44:32 debian kernel: [15884.715838] iwlwifi 0000:00:14.3: iwl_mvm_release_frames_from_notif Frame release notification for BAID 14, NSSN 169 >> Mar 4 12:44:32 debian kernel: [15884.715843] iwlwifi 0000:00:14.3: iwl_pcie_rx_handle_rb Q 0: RB end marker at offset 64 >> Mar 4 12:44:32 debian kernel: [15884.715852] iwlwifi 0000:00:14.3: iwl_pcie_restock_bd Assigned virtual RB ID 1348 to queue 0 index 334 >> >> - those events do not appear during normal operation (or very rarely) >> >> >> Temporary resolution: >> - in most cases, the STA remains in this state until Wifi is restarted or until it roams to another AP >> - while in that state, it may happens (rarely) that a few frame are received with very high latency, then the next ones are lost, for instance: >> >> [1646398334.114200] From 10.200.2.67 icmp_seq=148 Destination Host Unreachable >> [1646398334.114242] From 10.200.2.67 icmp_seq=149 Destination Host Unreachable >> [1646398334.114251] From 10.200.2.67 icmp_seq=150 Destination Host Unreachable >> [1646398336.365181] 64 bytes from 10.200.2.1: icmp_seq=151 ttl=64 time=2251 ms >> [1646398336.365237] 64 bytes from 10.200.2.1: icmp_seq=152 ttl=64 time=1227 ms >> [1646398336.365250] 64 bytes from 10.200.2.1: icmp_seq=153 ttl=64 time=203 ms >> [1646398375.042236] From 10.200.2.67 icmp_seq=188 Destination Host Unreachable >> [1646398375.042291] From 10.200.2.67 icmp_seq=189 Destination Host Unreachable >> [1646398375.042303] From 10.200.2.67 icmp_seq=190 Destination Host Unreachable >> >> >> Workaround: >> - disable_11ax=1 prevents the problem from happening >> [...] > >> Stephane Poignant 2022-03-10 14:48:39 UTC >> >> Did some further testing with vanilla kernel. >> 5.10.66 and older DO NOT reproduce the issue. >> 5.10.67 and newer DO reproduce. >> >> I see the following changes according to changelog: >> iwlwifi: mvm: Fix scan channel flags settings >> iwlwifi: fw: correctly limit to monitor dump >> iwlwifi: mvm: fix access to BSS elements >> iwlwifi: mvm: avoid static queue number aliasing >> iwlwifi: mvm: fix a memory leak in iwl_mvm_mac_ctxt_beacon_changed >> iwlwifi: pcie: free RBs during configure >> >> Suspecting the one related with queues but no strong opinion atm. >> >> [reply] [−] Comment 6 Stephane Poignant 2022-03-11 10:18:29 UTC >> >> Ok so after some further testing, turned out that after commenting the following lines in file drivers/net/wireless/intel/iwlwifi/pcie/trans.c: >> >> /* free all first - we might be reconfigured for a different size */ >> iwl_pcie_free_rbs_pool(trans); >> >> Which were introduced by the following commit: >> iwlwifi: pcie: free RBs during configure >> https://lore.kernel.org/all/iwlwifi.20210802170640.42d7c93279c4.I07f74e65aab0e3d965a81206fcb289dc92d74878@changeid/ >> >> Then i'm no longer able to reproduce. Tested in vanilla 5.10.67, vanilla 5.10.88 and 5.10.92 with Debian patches. >> > > Could somebody take a look into this? Or was this discussed somewhere > else already? Or even fixed? > > Anyway, to get this tracked: > > #regzbot introduced: 608c8359c567b4a04dedbe > #regzbot from: Stephane Poignant <stephane.poignant@xxxxxxxxx> > #regzbot title: wireless: iwlwifi: regression in 5.10.67 due to > "iwlwifi: pcie: free RBs during configure" > #regzbot link: https://bugzilla.kernel.org/show_bug.cgi?id=215660 > > Ciao, Thorsten (wearing his 'the Linux kernel's regression tracker' hat) > > P.S.: As the Linux kernel's regression tracker I'm getting a lot of > reports on my table. I can only look briefly into most of them and lack > knowledge about most of the areas they concern. I thus unfortunately > will sometimes get things wrong or miss something important. I hope > that's not the case here; if you think it is, don't hesitate to tell me > in a public reply, it's in everyone's interest to set the public record > straight. >
