Hello, [snip] > Finding 1: while it fails with > 1 GB RAM, using 512 MB or 1 GB works. > > Finding 2: with > 1 GB RAM, we're getting addresses over 4 GB in > sg->dma_address. We put some printks into arch/mips/mm/dma-coherent.c > and while everything looks okay with 1 GB of RAM, with 2 GB I get e.g. > map page a8000000063f36c0 to addr 000000017fc68000 > > That's an address > 4G. Lennert thinks that it's "most > likely going to stuff it into a 32bit address register > somewhere" and: "I can't see how it can work at all if it > passes the >32bit pci address to the device. it should > detect, hey, this is above 4G, so then allocate a buffer > below 4G, copy it into there, and pass _that_ buffer to the > device. that's called dma bounce buffering." Well, PCI supports both 64 and 32-bit addresses. There's a lot of PCI HW out there that's 32-bit only, but there's also a fair bit that isn't. A 32-bit device in a 64-bit system is going to require bounce buffering by the driver/OS. But, a 64-bit device won't (given a reasonable PCI controller implementation, which we have here). > Finding 3: the memory layout is weird. > memory: 000000000fe91e00 @ 0000000000000000 (usable) > memory: 000000001ffffe00 @ 0000000080000000 (usable) > memory: 000000000ffffe00 @ 00000000c0000000 (usable) > memory: 000000003ffffe00 @ 0000000140000000 (usable) > > Lennert, "if the pci controller doesn't compensate for such a > weird layout, you're bound to see pci issues." The PCI controller is built into the 1250/1480, and compensates for this. > Finding 4: the host bridge has some "unassigned" memory. Why? > > 0001:00:04.0 Host bridge: Broadcom Corporation: Unknown > device 0014 (rev 01) > Flags: bus master, fast devsel, latency 0, IRQ 255 > Memory at 60000000 (32-bit, prefetchable) [size=16M] > Memory at <unassigned> (32-bit, prefetchable) > Memory at 70000000 (32-bit, prefetchable) [size=4K] > Memory at <unassigned> (64-bit, prefetchable) Chuckle, I have *no* idea what this corresponds to. > > Does this information help? Also, we were wondering how to > find out whether a driver is okay with 64-bit dma addresses. This last part is the part I don't know about. While I've spent the last 12 years doing this sort of OS/driver programming, it wasn't under Linux and I'm still coming up to speed on the way things are handled there. IIRC most of the testing of the PCI code on the 1480 was done with a Broadcom PCI-X GigE ethernet card which does support 64-bit addresses, so the bounce buffering simply wouldn't have been an issue. I wouldn't be surprised if there are a number of 32-bit drivers out there that simply assume that they're always going to be in a 32-bit system. Unfortunately, the engineer who developed and tested this has since left, so I don't have a lot of details about what exactly was tested and what wasn't. HTH, Mark
