Hi Yuri,
On Thu, Dec 14, 2017 at 06:03:11PM +0300, Yuri Frolov wrote:
> Hi, James.
>
> Do I understand correctly, that in case of CONFIG_EVA=y, any logical
> address from 0x00000000 - 0xBFFFFFFF (3G) range accessed from the kernel
> space is:
> 1) Unmapped (no TLB translations)
> 2) Is mapped 1:1 to physical addresses? E.g, readl from 0x20000000 is,
> in fact a read from physical address 0x20000000? I mean, in legacy
> memory mapping scheme, logical addresses 0x80000000 - 0xBFFFFFFF in
> kernel space were being translated to the physical addresses from the
> low 512Mb (phys 0x00000000 - 0x20000000), no such bits stripping or
> something for EVA, the mapping is 1:1?
That depends on the EVA configuration. The hardware is fairly flexible
(which is both useful and painful - making supporting EVA from
multiplatform kernels particularly awkward), but that is one possible
configuration.
E.g. ideally you probably want to keep seg5 (0x00000000..0x3FFFFFFF)
MUSK (TLB mapped for user & kernel) so that null dereferences from
kernel code are safely caught, but that costs you 1GB of directly
accessible physical address space from kernel mode.
>
> As for userspace addresses, the addresses from the 0x00000000 -
> 0xBFFFFFFF range are:
> 1) Overlayed with the range which is directly accessible from the kernel
> space
Yes. Userland accesses access through tlb, kernel direct, and kernel eva
accesses (LWE/SWE etc) access through tlb from kernel
> 2) These addresses are mapped in userland, so, read from logical address
> 0x20000000 in userland may result in read from any physical address
> located in range 0x00000000 - 0xBFFFFFFF as defined by TLB for that
> particular userland thread?
Yes, though it could in theory be any physical address mapped by the
TLB, not just that range.
Cheers
James
>
> TIA,
> Yuri
>
> On 12/06/2017 02:46 PM, James Hogan wrote:
> > Hi Yuri,
> >
> > On Wed, Dec 06, 2017 at 01:36:48PM +0300, Yuri Frolov wrote:
> >> Hi,
> >>
> >> I'm trying to expand PCI mapped region using EVA addressing scheme for
> >> 'Baikal' P5600-based family of boards.
> >>
> >> Currently, we've got only one PCIe mapped region, 0x08000000 -
> >> 0x1BDBFFFF (~330Mb).
> >>
> >> This window is unsufficient for some PCIe cards, so I'm trying to expand
> >> (or add one more) PCIe mapped region by moving 'Hi Memory DRAM block'
> >> from 0x20000000 to 0x40000000 (dts changed, BAIKAL_HIGHMEM_START and
> >> HIGHMEM_START are changed in
> >> arch/mips/include/asm/mach-baikal/hardware.h
> >> /arch/mips/include/asm/mach-baikal/spaces.h respectively) and and
> >> allocating this region for PCIe.
> >>
> >> So far, PCI subsystem is initialized during the boot successfully (as
> >> far as I can see):
> >>
> >> dw_pcie_init: DEV_ID_VEND_ID=0x80601d39 CLASS_CODE_REV_ID=0x1
> >> dw_pcie_init: PCIe error code = 0x0
> >> dw_pcie_init: PCIe link speed GEN1
> >> device: 'pci0000:01': device_add
> >> PM: Adding info for No Bus:pci0000:01
> >> device: '0000:01': device_add
> >> PM: Adding info for No Bus:0000:01
> >> PCI host bridge to bus 0000:01
> >> pci_bus 0000:01: root bus resource [mem 0x20000000-0x37ffffff]
> >> pci_bus 0000:01: root bus resource [io 0x18020000-0x1bdaffff]
> >> pci_bus 0000:01: root bus resource [bus 01-ff]
> >> pci_bus 0000:01: scanning bus
> >> pci 0000:01:00.0: [8086:10d3] type 00 class 0x020000
> >> pci 0000:01:00.0: calling quirk_f0_vpd_link+0x0/0x98
> >> pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x0001ffff]
> >> pci 0000:01:00.0: reg 0x14: [mem 0x00000000-0x0007ffff]
> >> pci 0000:01:00.0: reg 0x18: [io 0x0000-0x001f]
> >> pci 0000:01:00.0: reg 0x1c: [mem 0x00000000-0x00003fff]
> >> pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0003ffff pref]
> >> pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
> >>
> >> pci 0000:01:00.0: PME# disabled
> >> device: '0000:01:00.0': device_add
> >> bus: 'pci': add device 0000:01:00.0
> >> PM: Adding info for pci:0000:01:00.0
> >> pci_bus 0000:01: fixups for bus
> >> pci_bus 0000:01: bus scan returning with max=01
> >> pci 0000:01:00.0: BAR 1: assigned [mem 0x20000000-0x2007ffff]
> >> pci 0000:01:00.0: BAR 6: assigned [mem 0x20080000-0x200bffff pref]
> >> pci 0000:01:00.0: BAR 0: assigned [mem 0x200c0000-0x200dffff]
> >> pci 0000:01:00.0: BAR 3: assigned [mem 0x200e0000-0x200e3fff]
> >> pci 0000:01:00.0: BAR 2: assigned [io 0x18020000-0x1802001f]
> >> pci 0000:01:00.0: calling quirk_e100_interrupt+0x0/0x20c
> >> pci 0000:01:00.0: calling baikal_t1_pcie_link_speed_fixup+0x0/0x228
> >> pci 0000:01:00.0: Link Capability is GEN1, x1
> >>
> >> but PCIe networking card (e1000e) driver fails to load:
> >>
> >> bus: 'pci': driver_probe_device: matched device 0000:01:00.0 with driver
> >> e1000e
> >> bus: 'pci': really_probe: probing driver e1000e with device 0000:01:00.0
> >> devices_kset: Moving 0000:01:00.0 to end of list
> >> PCI: Enabling device 0000:01:00.0 (0000 -> 0002)
> >> e1000e 0000:01:00.0: enabling bus mastering
> >> e1000_probe:7054 mmio_start 0x200c0000 mmio_len 0x20000
> >>
> >> e1000e 0000:01:00.0: Interrupt Throttling Rate (ints/sec) set to dynamic
> >> conservative mode
> >>
> >> PCI MSI: setup hwirq:0 virq:51
> >> PCI MSI: setup hwirq:1 virq:52
> >> PCI MSI: setup hwirq:2 virq:53
> >>
> >> e1000_get_phy_id_82571:444 ret_val = e1e_rphy(hw, MII_PHYSID1, &phy_id);
> >> ret_val = -2
> >>
> >> It turns out, that PHY ID can't be read from the memory, mapped on
> >> driver initialization function by ioremap.
> >>
> >> __ioremap (arch/mips/mm/ioremap.c) considers all the memory above the
> >> low 512Mb as DRAM and maps the physical addresses 0x20000000 -
> >> 0x3FFFFFFF to logical addresses starting from c0000000 and higher.
> >>
> >> arch/mips/include/asm/mach-baikal/kernel-entry-init.h defines EVA memory
> >> map with virtual addresses range 0xc0000000 - 0xdfffffff as 'MK (kseg2)'
> >> i.e. "mapped kernel" with cashing segment attributes defined in SegCtl0,
> >> Cfg1: "cacheable, coherent, write-back, write-allocate, read misses
> >> request shared attribute".
> >>
> >> I've tried to change the caching attribute (bits 16:18 in SegCtl0) to
> >> 0x2 ("uncached, not coherent"), but
> >>
> >> this hasn't yield any visible effect on PCIe e1000e initialization.
> > Mapped kernel segments don't use the CCA in SegCtl. Each page mapping
> > has its own CCA specified in the TLB, which would have been loaded from
> > page tables set up by e.g. remap_area_pte(), and the ioremap() macro
> > does pass _CACHE_UNCACHED as the flags argument. I would recommend
> > checking the phys_addr and flags coming in to remap_area_pte().
> >
> > If that all looks correct you could also try:
> > - Dump the TLB contents straight after accessing that address to check
> > they are loaded correctly, i.e. call dump_tlb_all() from
> > <asm/tlbdebug.h>.
> > - If the TLB contents look wrong, try disabling hardware page table
> > walker to determine whether that is misconfigured (add nohtw to kernel
> > command line).
> >
> > Hope that helps
> >
> > Cheers
> > James
>
Attachment:
signature.asc
Description: Digital signature
