On Thu, 24 Jul 2008 06:09:31 -0400
Prarit Bhargava <prarit@xxxxxxxxxx> wrote:
>
>
> Joerg Roedel wrote:
> > On Wed, Jul 23, 2008 at 07:47:03PM -0400, Prarit Bhargava wrote:
> >
> >>>> Interesting. Have you experienced any problems because of that
> >>>> misbehavior in the GART code? AMD IOMMU currently also violates this
> >>>> requirement. I will send a patch to fix that there too.
> >>>>
> >>>>
> >>>
> >>>
> >> Joerg, yes I can see misbehavior caused by this code. O/w I wouldn't
> >> be spending my time fixing it :) :)
> >>
> >> See below ....
> >>
> >>
> >>> IIRC, only PARISC and POWER IOMMUs follow the above rule. So I also
> >>> wondered what problem he hit.
> >>>
> >>>
> >> I wonder if IBM's Calgary IOMMU needs this fix? ... I've added Ed
> >> Pollard to find out.
> >>
> >> On big memory footprint (16G or above) systems it is possible that the
> >> e820 map reserves most of the lower 4G of memory for system use*. So
> >> it's possible that the 4G region is almost completely reserved at boot
> >> time and so the kernel starts using the IOMMU for DMA (see
> >> dma_alloc_coherent()). The addresses returned are not properly aligned,
> >> and this causes serious problems for some drivers that require a
> >> physical aligned address for the device.
> >>
> >
> > Do you have a list of driver which require this?
>
> No, I don't have a list. :(
>
> But it seems that the skge driver suffers from this because this code
> exists in the driver:
seems? You hit the bug with this driver, right?
> skge->mem = pci_alloc_consistent(hw->pdev, skge->mem_size,
> &skge->dma);
> if (!skge->mem)
> return -ENOMEM;
>
> BUG_ON(skge->dma & 7);
>
> if ((u64)skge->dma >> 32 != ((u64) skge->dma + skge->mem_size)
> >> 32) {
> printk(KERN_ERR PFX "pci_alloc_consistent region crosses
> 4G boundary\n");
> err = -EINVAL;
> goto free_pci_mem;
> }
>
>
> If pci_alloc_consistent did the "right" thing, we should *never* see
> that warning message.
Well, I think that this is not releated with the pci_alloc_consistent
alignment problem that you talk about.
I think that the driver tries to avoid 4GB boundary crossing
problem. You can find some work to avoid this, for example:
http://www.ussg.iu.edu/hypermail/linux/kernel/0712.0/2206.html
pci_device_add() has the following code to avoid this:
pci_set_dma_seg_boundary(dev, 0xffffffff);
I suspect that the problem you talk about, alloc_consistent doesn't
return the reqeuested size aligned memory, breaks anything.
> In theory, any 32-bit device attempting to request larger than PAGE_SIZE
> DMA memory on a system where no memory is available below 4G should show
> this problem.
>
> > I would like to
> > reproduce this issue. Does it also happen when you start the kernel with
> > iommu=force (GART should then be used for all DMA remapping) too?
> >
>
> Yes, this happens if you specify iommu=force on the command line.
>
> P.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
