Hi Marek,
On 10/02/2021 09:52, Neil Armstrong wrote:
> On 10/02/2021 09:45, Marek Szyprowski wrote:
>> Hi Neil,
>>
>> On 10.02.2021 09:17, Neil Armstrong wrote:
>>> On 09/02/2021 13:48, Marek Szyprowski wrote:
>>>> I've noticed that the Broadcom Wifi chip performs unaligned SDIO access
>>>> during the station scan on Khadas VIM3l board. This issue went unnoticed
>>>> so far, because there was a workaround in the meson MMC driver, which
>>>> has been recently disabled by commit e085b51c74cc ("mmc: meson-gx: check
>>>> for scatterlist size alignment in block mode") from current linux-next.
>>>>
>>>> I can easily reproduce this issue with the following commands:
>>>>
>>>> # dmesg | grep brcm
>>>> [ 11.659351] Bluetooth: hci0: BCM4359C0 'brcm/BCM4359C0.hcd' Patch
>>>> [ 13.079767] brcmfmac: brcmf_fw_alloc_request: using
>>>> brcm/brcmfmac4359-sdio for chip BCM4359/9
>>>> [ 13.527363] brcmfmac: brcmf_fw_alloc_request: using
>>>> brcm/brcmfmac4359-sdio for chip BCM4359/9
>>>> [ 13.601269] brcmfmac: brcmf_c_process_clm_blob: no clm_blob available
>>>> (err=-11), device may have limited channels available
>>>> [ 13.619414] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM4359/9 wl0:
>>>> Mar 6 2017 10:16:06 version 9.87.51.7 (r686312) FWID 01-4dcc75d9
>>>> # ifconfig wlan0 up
>>>> [ 208.052058] ieee80211 phy0: brcmf_dongle_roam: WLC_SET_ROAM_TRIGGER
>>>> error (-52)
>>>> # iw wlan0 scan >/dev/null
>>>> [ 218.148345] ------------[ cut here ]------------
>>>> [ 218.148501] unaligned sg len 504 blksize 256
>>>> [ 218.153712] WARNING: CPU: 1 PID: 75 at
>>>> drivers/mmc/host/meson-gx-mmc.c:251
>>>> meson_mmc_get_transfer_mode.isra.10+0xf8/0x130
>>>> [ 218.162616] Modules linked in: ipv6 brcmfmac brcmutil cfg80211
>>>> dw_hdmi_cec dw_hdmi_i2s_audio hci_uart btqca btbcm bluetooth
>>>> ecdh_generic ecc crct10dif_ce rfkill panfrost
>>>> snd_soc_meson_axg_sound_card snd_soc_meson_card_utils gpu_sched
>>>> meson_gxbb_wdt rtc_hym8563 pwm_meson meson_gxl rtc_meson_vrtc rc_khadas
>>>> meson_ir reset_meson_audio_arb realtek snd_soc_meson_g12a_tohdmitx
>>>> snd_soc_meson_codec_glue snd_soc_meson_axg_tdmout
>>>> snd_soc_meson_axg_frddr snd_soc_meson_axg_fifo axg_audio sclk_div
>>>> clk_phase mdio_mux_meson_g12a meson_dw_hdmi meson_drm meson_rng
>>>> dwmac_generic snd_soc_meson_axg_tdm_interface meson_canvas rng_core
>>>> dw_hdmi snd_soc_meson_axg_tdm_formatter dwmac_meson8b stmmac_platform
>>>> stmmac display_connector adc_keys pcs_xpcs nvmem_meson_efuse
>>>> [ 218.228329] CPU: 1 PID: 75 Comm: kworker/u8:2 Not tainted
>>>> 5.11.0-rc6-next-20210208 #2492
>>>> [ 218.236343] Hardware name: Khadas VIM3L (DT)
>>>> [ 218.240579] Workqueue: brcmf_wq/mmc2:0001:1 brcmf_sdio_dataworker
>>>> [brcmfmac]
>>>> [ 218.247559] pstate: 60400009 (nZCv daif +PAN -UAO -TCO BTYPE=--)
>>>> [ 218.253506] pc : meson_mmc_get_transfer_mode.isra.10+0xf8/0x130
>>>> [ 218.259372] lr : meson_mmc_get_transfer_mode.isra.10+0xf8/0x130
>>>> [ 218.265237] sp : ffff80001327b870
>>>> [ 218.268514] x29: ffff80001327b870 x28: 0000000000000003
>>>> [ 218.273776] x27: 0000000000000218 x26: 0000000000000600
>>>> [ 218.279036] x25: ffff000003a17678 x24: 0000000000000000
>>>> [ 218.284296] x23: ffff8000121cfae8 x22: ffff800011b54000
>>>> [ 218.289559] x21: ffff80001327bb18 x20: ffff00000277d000
>>>> [ 218.294819] x19: ffff80001327ba40 x18: 00000002faf07f80
>>>> [ 218.300081] x17: 0000000000004000 x16: 0000000000000000
>>>> [ 218.305341] x15: 0000000000000380 x14: 0000000000000000
>>>> [ 218.310603] x13: 0000000000000080 x12: 0000000000000000
>>>> [ 218.315865] x11: 0000000000000000 x10: 0000000000001370
>>>> [ 218.321125] x9 : ffff00006f995258 x8 : 000000009e0e7ca7
>>>> [ 218.326387] x7 : ffff80001327b4a0 x6 : 0000000000000001
>>>> [ 218.331648] x5 : 0000000000000001 x4 : 0000000000000000
>>>> [ 218.336912] x3 : 0000000000000002 x2 : ffff8000121f4768
>>>> [ 218.342171] x1 : 42fe1bd05e5eaa00 x0 : 0000000000000000
>>>> [ 218.347435] Call trace:
>>>> [ 218.349854] meson_mmc_get_transfer_mode.isra.10+0xf8/0x130
>>>> [ 218.355368] meson_mmc_request+0x74/0xb0
>>>> [ 218.359250] __mmc_start_request+0xa4/0x2b0
>>>> [ 218.363390] mmc_start_request+0x80/0xa8
>>>> [ 218.367272] mmc_wait_for_req+0x68/0xd8
>>>> [ 218.371066] mmc_submit_one.isra.17+0x78/0x148 [brcmfmac]
>>>> [ 218.376416] brcmf_sdiod_sglist_rw+0x324/0x4a8 [brcmfmac]
>>>> [ 218.381762] brcmf_sdiod_recv_chain+0x70/0x140 [brcmfmac]
>>>> [ 218.387110] brcmf_sdio_dataworker+0x614/0x17d0 [brcmfmac]
>>>> [ 218.392545] process_one_work+0x2a8/0x728
>>>> [ 218.396510] worker_thread+0x48/0x460
>>>> [ 218.400132] kthread+0x134/0x160
>>>> [ 218.403323] ret_from_fork+0x10/0x18
>>>> [ 218.406862] irq event stamp: 15834
>>>> [ 218.410222] hardirqs last enabled at (15833): [<ffff800010f95804>]
>>>> _raw_spin_unlock_irq+0x3c/0x80
>>>> [ 218.419107] hardirqs last disabled at (15834): [<ffff800010f895ac>]
>>>> el1_dbg+0x24/0x50
>>>> [ 218.426869] softirqs last enabled at (15828): [<ffff800010010508>]
>>>> _stext+0x508/0x638
>>>> [ 218.434717] softirqs last disabled at (15823): [<ffff8000100952ec>]
>>>> irq_exit+0x19c/0x1a8
>>>> [ 218.442739] ---[ end trace dfc38bb4458b4c37 ]---
>>>> #
>>>>
>>>> Surprisingly the same commands executed on the Khadas VIM3 board with
>>>> the same kernel don't trigger the warning.
>>> This may be because the VIM3 (G12A & G12B) has a broken SDIO controller using
>>> an internal SRAM as bounce buffer instead of the scatter/gather DMA, cf amlogic,dram-access-quirk.
>> Good to know, this explains why it works without an issue on VIM3.
>>>> Let me know if I can help debugging this issue.
>>> Simply remove the WARN_ONCE()...
>>
>> Well, that WARN is not a big issue for me. I just wanted to help fixing
>> the real issue. If I understand right, the brcmfmac driver queues
>> incorrectly prepared sd list for the given transfer mode, what should be
>> fixed there.
>
> I think the issue comes from brcmfmac that should not provide unaligned data,
> but I think it's a global issue where these kind of details aren't specified/enforced.
>
>>
>> On the other hand, if this is just a limitation of the meson mmc
>> controller on SM1/VIM3l, then imho there should not be a WARN there and
>> maybe the mentioned 'amlogic,dram-access-quirk' could be used to fix the
>> issue on VIM3l too. I've checked and it indeed 'fixes' the issue on that
>> board.
>
> This fix severely limits the performance so it's not a good fix here,
> the only fix I see here is to remove the WARN because the fact is that the DMA cannot handle
> unaligned data, and the SDIO device drivers should be fixed in order to achieve max
> performance on SDIO host drivers with DMA (it's usual to not support unaligned data
> when dealing with a DMA).
Do you plan to remove the warning ? I can push a quick patch if not.
>
> Neil
>
>>
>> Best regards
>>
>
