Hi, I'm trying to understand how these flags are supposed to work. What I'd like to do, is when my pcie device runtime suspends (d3cold), I would like it to stay off during system suspend and resume and on be turned on again if the user accesses the device which would cause it to be runtime resumed. Runtime pm works fine on the devices, but they get resumed on system suspend and powered back up on system resume. The driver provides the following pm callbacks: static const struct dev_pm_ops amdgpu_pm_ops = { .suspend = amdgpu_pmops_suspend, .resume = amdgpu_pmops_resume, .freeze = amdgpu_pmops_freeze, .thaw = amdgpu_pmops_thaw, .poweroff = amdgpu_pmops_poweroff, .restore = amdgpu_pmops_restore, .runtime_suspend = amdgpu_pmops_runtime_suspend, .runtime_resume = amdgpu_pmops_runtime_resume, .runtime_idle = amdgpu_pmops_runtime_idle, }; I'm having trouble parsing the last few paragraphs in the pm documentation (https://www.kernel.org/doc/html/latest/driver-api/pm/devices.html). Ideally once the device is runtime suspended, it would not be powered up again until someone accesses the device and it runtime resumes with possibly multiple system suspend/resume cycles in between. "If that function returns true, the driver’s “noirq” and “early” resume callbacks should be skipped and the device’s runtime PM status will be set to “suspended” by the PM core. Otherwise, if the device was runtime-suspended during the preceding system-wide suspend transition and its DPM_FLAG_SMART_SUSPEND is set, its runtime PM status will be set to “active” by the PM core. [Hence, the drivers that do not set DPM_FLAG_SMART_SUSPEND should not expect the runtime PM status of their devices to be changed from “suspended” to “active” by the PM core during system-wide resume-type transitions.]" The last sentence is confusing me. Why would setting DPM_FLAG_SMART_SUSPEND change the status to active? Wouldn't still be suspended? "Likewise, if DPM_FLAG_MAY_SKIP_RESUME is set for a device, its driver’s system-wide “noirq” and “early” resume callbacks may be skipped while its “late” and “noirq” suspend callbacks may have been executed (in principle, regardless of whether or not DPM_FLAG_SMART_SUSPEND is set). In that case, the driver needs to be able to cope with the invocation of its ->runtime_resume callback back-to-back with its “late” and “noirq” suspend ones. [For instance, that is not a concern if the driver sets both DPM_FLAG_SMART_SUSPEND and DPM_FLAG_MAY_SKIP_RESUME and uses the same pair of suspend/resume callback functions for runtime PM and system-wide suspend/resume.]" The last sentence here is confusing me again. I'm not following why it matters if the callbacks are the same. Why would the suspend and resume callbacks be called at all. Wouldn't it just be the runtime variants? It's not clear what happens with the suspend and resume pm callbacks. The documentation only mentions the early/late and noirq callbacks. Is the idea that the driver should not use the suspend/resume callbacks in this case if they want to skip system suspend/resume in favor of the runtime variant? Thanks, Alex