On 31/03/2021 12:41, Joakim Zhang wrote: ... >> In answer to your question, resuming from suspend does work on this board >> without your change. We have been testing suspend/resume now on this board >> since Linux v5.8 and so we have the ability to bisect such regressions. So it is >> clear to me that this is the change that caused this, but I am not sure why. > > Yes, I know this issue is regression caused by my patch. I just want to analyze the potential reasons. Due to the code change only related to the page recycle and reallocate. > So I guess if this page operate need IOMMU works when IOMMU is enabled. Could you help check if IOMMU driver resume before STMMAC? Our common desire is to find the root cause, right? Yes of course that is the desire here indeed. I had assumed that the suspend/resume order was good because we have never seen any problems, but nonetheless it is always good to check. Using ftrace I enabled tracing of the appropriate suspend/resume functions and this is what I see ... # tracer: function # # entries-in-buffer/entries-written: 4/4 #P:6 # # _-----=> irqs-off # / _----=> need-resched # | / _---=> hardirq/softirq # || / _--=> preempt-depth # ||| / delay # TASK-PID CPU# |||| TIMESTAMP FUNCTION # | | | |||| | | rtcwake-748 [000] ...1 536.700777: stmmac_pltfr_suspend <-platform_pm_suspend rtcwake-748 [000] ...1 536.735532: arm_smmu_pm_suspend <-platform_pm_suspend rtcwake-748 [000] ...1 536.757290: arm_smmu_pm_resume <-platform_pm_resume rtcwake-748 [003] ...1 536.856771: stmmac_pltfr_resume <-platform_pm_resume So I don't see any ordering issues that could be causing this. Jon -- nvpublic