On Tue, 30 Apr 2024 at 13:20, Robin Murphy <robin.murphy@xxxxxxx> wrote: > > On 2024-04-30 1:41 am, Dmitry Baryshkov wrote: > > On Tue, 30 Apr 2024 at 01:26, Robin Murphy <robin.murphy@xxxxxxx> wrote: > >> > >> On 2024-04-29 5:31 pm, Dmitry Baryshkov wrote: > >>> On Fri, Apr 19, 2024 at 05:54:45PM +0100, Robin Murphy wrote: > >>>> It's somewhat hard to see, but arm64's arch_setup_dma_ops() should only > >>>> ever call iommu_setup_dma_ops() after a successful iommu_probe_device(), > >>>> which means there should be no harm in achieving the same order of > >>>> operations by running it off the back of iommu_probe_device() itself. > >>>> This then puts it in line with the x86 and s390 .probe_finalize bodges, > >>>> letting us pull it all into the main flow properly. As a bonus this lets > >>>> us fold in and de-scope the PCI workaround setup as well. > >>>> > >>>> At this point we can also then pull the call up inside the group mutex, > >>>> and avoid having to think about whether iommu_group_store_type() could > >>>> theoretically race and free the domain if iommu_setup_dma_ops() ran just > >>>> *before* iommu_device_use_default_domain() claims it... Furthermore we > >>>> replace one .probe_finalize call completely, since the only remaining > >>>> implementations are now one which only needs to run once for the initial > >>>> boot-time probe, and two which themselves render that path unreachable. > >>>> > >>>> This leaves us a big step closer to realistically being able to unpick > >>>> the variety of different things that iommu_setup_dma_ops() has been > >>>> muddling together, and further streamline iommu-dma into core API flows > >>>> in future. > >>>> > >>>> Reviewed-by: Lu Baolu <baolu.lu@xxxxxxxxxxxxxxx> # For Intel IOMMU > >>>> Reviewed-by: Jason Gunthorpe <jgg@xxxxxxxxxx> > >>>> Tested-by: Hanjun Guo <guohanjun@xxxxxxxxxx> > >>>> Signed-off-by: Robin Murphy <robin.murphy@xxxxxxx> > >>>> --- > >>>> v2: Shuffle around to make sure the iommu_group_do_probe_finalize() case > >>>> is covered as well, with bonus side-effects as above. > >>>> v3: *Really* do that, remembering the other two probe_finalize sites too. > >>>> --- > >>>> arch/arm64/mm/dma-mapping.c | 2 -- > >>>> drivers/iommu/amd/iommu.c | 8 -------- > >>>> drivers/iommu/dma-iommu.c | 18 ++++++------------ > >>>> drivers/iommu/dma-iommu.h | 14 ++++++-------- > >>>> drivers/iommu/intel/iommu.c | 7 ------- > >>>> drivers/iommu/iommu.c | 20 +++++++------------- > >>>> drivers/iommu/s390-iommu.c | 6 ------ > >>>> drivers/iommu/virtio-iommu.c | 10 ---------- > >>>> include/linux/iommu.h | 7 ------- > >>>> 9 files changed, 19 insertions(+), 73 deletions(-) > >>> > >>> This patch breaks UFS on Qualcomm SC8180X Primus platform: > >>> > >>> > >>> [ 3.846856] arm-smmu 15000000.iommu: Unhandled context fault: fsr=0x402, iova=0x1032db3e0, fsynr=0x130000, cbfrsynra=0x300, cb=4 > >> > >> Hmm, a context fault implies that the device did get attached to a DMA > >> domain, thus has successfully been through __iommu_probe_device(), yet > >> somehow still didn't get the right DMA ops (since that "IOVA" looks more > >> like a PA to me). Do you see the "Adding to IOMMU group..." message for > >> this device, and/or any other relevant messages or errors before this > >> point? > > > > No, nothing relevant. > > > > [ 8.372395] ufshcd-qcom 1d84000.ufshc: Adding to iommu group 6 > > > > (please ignore the timestamp, it comes before ufshc being probed). > > > >> I'm guessing there's a fair chance probe deferral might be > >> involved as well. I'd like to understand what path(s) this ends up > >> taking through __iommu_probe_device() and of_dma_configure(), or at > >> least the number and order of probe attempts between the UFS and SMMU > >> drivers. > > > > __iommu_probe_device() gets called twice and returns early because ops is NULL. > > > > Then finally of_dma_configure_id() is called. The following branches are taken: > > > > np == dev->of_node > > of_dma_get_range() returned 0 > > bus_dma_limit and dma_range_map are set > > __iommu_probe_device() is called, using the `!group->default_domain && > > !group_lis` case, then group->default_domain() is not NULL, > > In the end, iommu_setup_dma_ops() is called. > > > > Then the ufshc probe defers (most likely the PHY is not present or > > some other device is not there yet). > > Ah good, probe deferral. And indeed the half-formed hunch from last > night grew into a pretty definite idea by this morning... patch incoming. Thanks a lot for the quick fix! -- With best wishes Dmitry
