Hi Kishon, On Sat, Mar 28, 2020 at 6:44 PM Lad, Prabhakar <prabhakar.csengg@xxxxxxxxx> wrote: > > Hi Kishon, > > On Tue, Mar 24, 2020 at 2:41 PM Lad, Prabhakar > <prabhakar.csengg@xxxxxxxxx> wrote: > > > > Hi Kishon, > > > > On Tue, Mar 24, 2020 at 1:58 AM Kishon Vijay Abraham I <kishon@xxxxxx> wrote: > > > > > > Hi Prabhakar, > > > > > > On 3/22/2020 4:19 AM, Lad, Prabhakar wrote: > > > > Hi Kishon, > > > > > > > > On Fri, Mar 20, 2020 at 5:28 AM Kishon Vijay Abraham I <kishon@xxxxxx> wrote: > > > >> > > > >> Hi Prabhakar, > > > >> > > > >> On 3/18/2020 5:07 PM, Lad, Prabhakar wrote: > > > >>> Hi Kishon, > > > >>> > > > >>> I rebased my rcar-endpoint patches on endpoint branch, which has > > > >>> support for streaming DMA API support, with this read/write/copy > > > >>> tests failed, to make sure nothing hasn't changed on my driver I > > > >>> reverted the streaming DMA API patch > > > >>> 74b9b4da84c71418ceeaaeb78dc790376df92fea "misc: pci_endpoint_test: Use > > > >>> streaming DMA APIs for buffer allocation" and tests began to pass > > > >>> again. > > > >>> > > > >>> If add a GFP_DMA flag for kzalloc (with streaming DMA), the test cases > > > >>> for read/write/copy pass as expected. > > > >>> > > > >>> Could you please through some light why this could be happening. > > > >> > > > >> Do you see any differences in the address returned by dma_map_single() like is > > > >> it 32-bit address or 64-bit address? > > > >> > > > > Both return 32 bit address, debugging further I see that with > > > > GFP_KERNEL flag for small buffer > > > > sizes the read/write/copy tests pass(upto 4k), so I am suspecting its > > > > related to caching probably. > > > > Also adding wmb()/rmb() just with GFP_KERNEL flag didn't help. Note I > > > > am using PIO transfers. > > > > Any thoughts on how we tackle it ? > > > > > > > > # With GFP_KERNEL flag > > > > root@hihope-rzg2m:~# pcitest -r > > > > [ 46.210649] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_read > > > > kzalloc:ffff0004b4ae0000 dma:7e99d000 align:ffff0004b4ae0000 > > > > READ ( 102400 bytes): NOT OKAY > > > > root@hihope-rzg2m:~# pcitest -r > > > > [ 51.880063] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_read > > > > kzalloc:ffff0004b4ae0000 dma:7e9c0000 align:ffff0004b4ae0000 > > > > READ ( 102400 bytes): OKAY > > > > > > Here one of the read test is passing and the other is failing. > > > For the 1st case dma:7e99d000, address is aligned to 4K > > > For the 2nd case dma:7e9c0000, address is aligned to 256K > > > > > > I'm suspecting this could be an alignment issue. Does the outbound ATU of your > > > EP has any restrictions? (like the address should be aligned to the size?). > > > > > There isn't any restriction for outbound ATU on ep, Although I tried > > alignment from > > SZ_1 - SZ_256K and each failed at several points. > > > > With GFP_KERNEL | GFP_DMA, as in my previous dump here the address too > > is not aligned to 256 but still read passes. > > root@hihope-rzg2m:~# pcitest -r -s 16384 > > [ 186.629347] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_read > > kzalloc:ffff00003b848000 dma:7b848000 align:ffff00003b848000 > > READ ( 16384 bytes): OKAY > > > > And I have verified with GFP_KERNEL | GFP_DMA on my platform > > everything works as expected, > > > > So how about a patch for pci_endpoint_test.c, where flags are passed > > as part of driver_data and it defaults to just GFP_KERNEL ? > > > Any thoughts on the above ? I intended to get the endpoint driver for v5.7. > Correct me if I am wrong here, streaming DMA API should be used with dma (-d) option so that root device makes sure the data is synced when data is transferred whereas previously with dma_alloc_coherent() we didn't have to care about cache issues. Also for a non-dma (-d) option we don't have a handle to dma in rootpport device so that we can call a sync operation. I say this because on my platform with streaming DMA api it works for small size buffers but it doesn't work with large size buffers. Could you please confirm with streaming DMA api without DMA (-d) option for large buffers read/write/copy still passes for you. Although I am not sure why adding GFP_KERNEL | GFP_DMA flag for kzalloc on my platform fixes everything. Cheers, --Prabhakar > Cheers, > --Prabhakar > > > Cheers, > > --Prabhakar > > > > > Thanks > > > Kishon > > > > > > > root@hihope-rzg2m:~# pcitest -r > > > > [ 53.354830] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_read > > > > kzalloc:ffff0004b4ae0000 dma:7e9e2000 align:ffff0004b4ae0000 > > > > READ ( 102400 bytes): NOT OKAY > > > > root@hihope-rzg2m:~# pcitest -r > > > > [ 55.307236] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_read > > > > kzalloc:ffff0004b4ae0000 dma:7ea04000 align:ffff0004b4ae0000 > > > > READ ( 102400 bytes): NOT OKAY > > > > root@hihope-rzg2m:~# pcitest -r > > > > [ 57.098626] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_read > > > > kzalloc:ffff0004b4ae0000 dma:7ea23000 align:ffff0004b4ae0000 > > > > READ ( 102400 bytes): NOT OKAY > > > > > > > > # GFP_KERNEL | GFP_DMA > > > > > > > > root@hihope-rzg2m:~# pcitest -r -s 1024001 > > > > [ 174.562071] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_read > > > > kzalloc:ffff00003b900000 dma:7b900000 align:ffff00003b900000 > > > > READ (1024001 bytes): OKAY > > > > root@hihope-rzg2m:~# pcitest -r -s 16384 > > > > [ 186.629347] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_read > > > > kzalloc:ffff00003b848000 dma:7b848000 align:ffff00003b848000 > > > > READ ( 16384 bytes): OKAY > > > > root@hihope-rzg2m:~# pcitest -r -s 8192 > > > > [ 190.578335] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_read > > > > kzalloc:ffff00003b840000 dma:7b840000 align:ffff00003b840000 > > > > READ ( 8192 bytes): OKAY > > > > root@hihope-rzg2m:~# pcitest -r -s 128 > > > > [ 199.428021] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_read > > > > kzalloc:ffff00003b800000 dma:7b800000 align:ffff00003b800000 > > > > READ ( 128 bytes): OKAY > > > > root@hihope-rzg2m:~# > > > > > > > > Cheers, > > > > --Prabhakar > > > > > > > >> Thanks > > > >> Kishon
