On 2019-02-22 8:45 a.m., Thomas Hellstrom wrote: > On Fri, 2019-02-22 at 07:10 +0000, Koenig, Christian wrote: >> Am 21.02.19 um 22:02 schrieb Thomas Hellstrom: >>> Hi, >>> >>> On Thu, 2019-02-21 at 20:24 +0000, Kuehling, Felix wrote: >>>> On 2019-02-21 12:34 p.m., Thomas Hellstrom wrote: >>>>> On Thu, 2019-02-21 at 16:57 +0000, Kuehling, Felix wrote: >>>>>> On 2019-02-21 2:59 a.m., Koenig, Christian wrote: >>>>>>> On x86 with HIGHMEM there is no dma32 zone. Why do we need >>>>>>> one >>>>>>> on >>>>>>>>> x86_64? Can we make x86_64 more like HIGHMEM instead? >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Felix >>>>>>>>> >>>>>>>> IIRC with x86, the kernel zone is always smaller than any >>>>>>>> dma32 >>>>>>>> zone, >>>>>>>> so we'd always exhaust the kernel zone before dma32 >>>>>>>> anyway. >>>>>>>> >>>>>>>> Not sure why we have dma32 on x86 without highmem, >>>>>>>> though. >>>>>>>> sounds >>>>>>>> superflous but harmless. >>>>>>> Well DMA32 denotes memory which is accessible by devices >>>>>>> who >>>>>>> can >>>>>>> only do >>>>>>> 32bit addressing. And IIRC we can actually do DMA32 to >>>>>>> highmem >>>>>>> since >>>>>>> something like 2.4.*. >>>>>>> >>>>>>> Because of this it is actually irrelevant if you have >>>>>>> highmem >>>>>>> or >>>>>>> not, >>>>>>> what matters for DMA32 is if you have an IOMMU or not. >>>>>> Are you saying we should have a dma32_zone even on x86 with >>>>>> HIGHMEM? >>>>>> >>>>>> >>>>>>> So even on x86_64 you actually do need the DMA32 zone if >>>>>>> you >>>>>>> don't >>>>>>> have >>>>>>> an IOMMU which remaps all memory for devices which can't >>>>>>> directly >>>>>>> address it. >>>>>> Why is DMA32 special in this way? For example AMD GFX8 GPUs >>>>>> support >>>>>> 40-bit DMA. But we don't have a special zone for that. >>>>> If you're running on a non-IOMMU system with physical memory >>>>> addresses >>>>>> 40 bits, and tell the DMA subsystem that you need to restrict >>>>>> to >>>>>> 40 >>>>> bits, it will probably start using bounce buffers for streaming >>>>> DMA >>>>> (which won't work with most graphics drivers), or for >>>>> dma_alloc_coherent(), it will probably use memory from the >>>>> DMA32 >>>>> zone. >>>> OK, then why is it not needed when CONFIG_HIGHMEM is defined? >>>> >>>> I found that there is a CONFIG_ZONE_DMA32 parameter. Maybe we >>>> should >>>> use >>>> that to decide whether to account for the DMA32 zone in TTM. It >>>> is >>>> set >>>> on x86_64 and a number of other 64-bit architectures. >>>> >>>> >>>>>> How common is it to have devices that need DMA32 on an x86_64 >>>>>> system? >>>>> IIRC All devices using dma_alloc_coherent() allocate DMA32 >>>>> memory >>>>> unless they explicitly set the dma coherent mask to something >>>>> larger. >>>>> Like Christian says, if an IOMMU is present and enabled, the >>>>> need >>>>> for >>>>> the DMA32 zone goes away. In theory at least. >>>> Thanks. I read up a bit on DMA32 and memory zones in general. I >>>> found >>>> that there is a lowmem_reserve_ratio feature that protects >>>> against >>>> normal page allocations overflowing into lowmem zones. There is >>>> some >>>> documentation in Documentation/scsctl/vm.txt (search for >>>> lowmem_reserve_ratio). The protected amount of memory in each >>>> zone >>>> can >>>> be seen in /proc/zoneinfo. >>>> >>>> With that, can we conclude that we don't need to count >>>> ttm_mem_global_alloc against the dma32 zone. >>> Yes, it indeed looks like that. >>> But then I would suggest removing the DMA32 zone entirely. >> We still need it for the pages we allocate, but we should just stop >> accounting all the housekeeping to it. > Why is that? Can't we just account all pages in the kernel zone, and > leave it up to the kernel to make sure there are still DMA32 pages > left? ttm_page_alloc and ttm_page_alloc_dma support allocating from DMA32 explicitly (setting GFP_DMA32). Such allocations could exhaust DMA32 memory, which TTM should prevent by limiting its DMA32 usage. This would still be counted against the dma32 zone by ttm_mem_global_alloc_page. I'll send out a new patch that counts general kernel allocations against the kernel zone only. I hope this would be acceptable. Regards, Felix > > /Thomas > _______________________________________________ amd-gfx mailing list amd-gfx@xxxxxxxxxxxxxxxxxxxxx https://lists.freedesktop.org/mailman/listinfo/amd-gfx
