On 7/28/22 10:45, Tudor.Ambarus@xxxxxxxxxxxxx wrote: > EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe > > On 7/13/22 19:01, Tudor.Ambarus@xxxxxxxxxxxxx wrote: >> EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe >> >> Hi, Peter, >> >> Thanks for the patience. I was still out of office last week, >> but now I have some news. >> >> On 6/27/22 19:53, Tudor.Ambarus@xxxxxxxxxxxxx wrote: >>> I think these are the last less invasive >>> changes that I try, I'll have to rewrite the logic anyway. >> >> I've chopped the driver to use virt-dma (check [1]). It's not clean, but >> it works and one can see how the logic is changed. Unfortunately the mem >> corruption is still present on high loads. Maybe it's a coherency problem. >> I need more time on it. Will get back to you. >> >> Cheers, >> ta >> >> [1] To github.com:ambarus/linux-0day.git >> a7351e6f4c12..1557e0df0fd0 at-hdmac-virt-dma -> at-hdmac-virt-dma > > Hi, Peter, > > Does this [1] one line patch solve the mem corruption on your side? > Even if yes, there are still bugs in at-hdmac that can be squashed by > using virt-dma. I'd like to follow up with patches that integrate > virt-dma logic in at-hdmac. > > Cheers, > ta > > [1] https://lore.kernel.org/linux-mtd/20220728074014.145406-1-tudor.ambarus@xxxxxxxxxxxxx/T/#u Hi, Peter, Looks like I've already caught an oops in at-hdmac driver when not using virt-dma, see below. Would you please test with all the patches from [2] instead of just using the patch from [1]? I've run stress tests over night by using [2] and everything went fine on my side. Cheers, ta [2] To github.com:ambarus/linux-0day.git * [new branch] at-hdmac-virt-dma-2nd-iteration -> at-hdmac-virt-dma-2nd-iteration root@sama5d3-xplained:~# while :; do cat testfile | sha256sum; done dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - dad5c65bf4c2a009ad6bad0d279096841de91631636057dd5dc47d0a45f9ec84 - [ 4115.100000] ------------[ cut here ]------------ [ 4115.100000] kernel BUG at drivers/dma/dmaengine.h:54! [ 4115.100000] Internal error: Oops - BUG: 0 [#1] ARM [ 4115.100000] CPU: 0 PID: 480 Comm: cat Not tainted 5.18.0-rc7+ #40 [ 4115.100000] Hardware name: Atmel SAMA5 [ 4115.100000] PC is at atc_chain_complete+0x150/0x168 [ 4115.100000] LR is at atc_advance_work+0x78/0x184 [ 4115.100000] pc : [<c03d1fe4>] lr : [<c03d21d4>] psr: 60030093 [ 4115.100000] sp : c4e597e8 ip : c0d72800 fp : 00000001 [ 4115.100000] r10: c4e59848 r9 : c17c4300 r8 : 60030013 [ 4115.100000] r7 : c0d72c58 r6 : c17c4300 r5 : c0d72c88 r4 : c0d72be8 [ 4115.100000] r3 : 00000000 r2 : 00000000 r1 : c0d72c3c r0 : c0d72be8 [ 4115.100000] Flags: nZCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment none [ 4115.100000] Control: 10c53c7d Table: 20ae0059 DAC: 00000051 [ 4115.100000] Register r0 information: slab kmalloc-2k start c0d72800 pointer offset 1000 size 2048 [ 4115.100000] Register r1 information: slab kmalloc-2k start c0d72800 pointer offset 1084 size 2048 [ 4115.100000] Register r2 information: NULL pointer [ 4115.100000] Register r3 information: NULL pointer [ 4115.100000] Register r4 information: slab kmalloc-2k start c0d72800 pointer offset 1000 size 2048 [ 4115.100000] Register r5 information: slab kmalloc-2k start c0d72800 pointer offset 1160 size 2048 [ 4115.100000] Register r6 information: slab task_struct start c17c4300 pointer offset 0 [ 4115.100000] Register r7 information: slab kmalloc-2k start c0d72800 pointer offset 1112 size 2048 [ 4115.100000] Register r8 information: non-paged memory [ 4115.100000] Register r9 information: slab task_struct start c17c4300 pointer offset 0 [ 4115.100000] Register r10 information: 2-page vmalloc region starting at 0xc4e58000 allocated at kernel_clone+0xb4/0x358 [ 4115.100000] Register r11 information: non-paged memory [ 4115.100000] Register r12 information: slab kmalloc-2k start c0d72800 pointer offset 0 size 2048 [ 4115.100000] Process cat (pid: 480, stack limit = 0x511e7a27) [ 4115.100000] Stack: (0xc4e597e8 to 0xc4e5a000) [ 4115.100000] 97e0: c4e59848 00000001 00000010 4f21ffb9 c0d72be8 c0d72c88 [ 4115.100000] 9800: c17c4300 00000002 00000800 c03d21d4 00000800 c17c4300 c4e59848 4f21ffb9 [ 4115.100000] 9820: 00000010 c0e7d100 00000000 20f28000 00000002 00000800 c17c4300 c0467490 [ 4115.100000] 9840: 00000003 00000000 00000001 c4e5984c c4e5984c 4f21ffb9 c0e6e040 c0f28000 [ 4115.100000] 9860: 00000800 c0e7d100 c090e8bc c090e99c c0b41d18 c0467688 00000002 00000000 [ 4115.100000] 9880: c4e59888 00000000 00000000 c4e59990 c08f4bd4 00000014 00000001 c04677fc [ 4115.100000] 98a0: c4e59990 c045a918 00000000 00000000 00000000 00000004 c0833744 00000001 [ 4115.100000] 98c0: 00000000 00000000 00000001 c4e59990 c4e59990 c4e59990 00000001 c0832df4 [ 4115.100000] 98e0: c0b41d18 00000000 c4e59984 c0e6e050 c17c4300 c090e904 c4e59990 00000001 [ 4115.100000] 9900: 00000000 c4e59990 00000001 00000000 00000000 c4e59990 00000001 00000000 [ 4115.100000] 9920: c4e59990 00000001 00000000 00000000 c4e59990 00000001 00000000 c4e59990 [ 4115.100000] 9940: 00000000 00000000 00000000 4f21ffb9 00000200 00000000 c0e6e050 c17c4300 [ 4115.100000] 9960: c0f28000 c0464d54 00000800 c0831be0 c4e59ac4 c04595b4 c17c4300 00000000 [ 4115.100000] 9980: c4e59ac4 00000000 c4e59990 00000001 00000002 00000800 c0f28000 4f21ff00 [ 4115.100000] 99a0: 00000000 4f21ffb9 00000000 c0e6e050 00000000 c0f28000 00000000 c0467b38 [ 4115.100000] 99c0: 00000000 c0466dc4 00000000 ffffffff c0e6e050 c0e6e050 00000480 00000000 [ 4115.100000] 99e0: 00000000 c0467b54 00000000 00000000 c4e59ac4 c04569b4 00001030 00000000 [ 4115.100000] 9a00: 00000000 00000000 c4e59a4c c0f28000 00000800 00000001 0001e872 c0e6e12c [ 4115.100000] 9a20: 00000000 c1704bb0 c0f28000 00000480 00000000 0001e872 00000000 00000000 [ 4115.100000] 9a40: 00000000 00000040 00000000 c0b41d18 00000134 00000000 00000002 00000000 [ 4115.100000] 9a60: 00000000 c0e6e050 c0f67000 0ec38450 00000000 00000134 00000000 00000000 [ 4115.100000] 9a80: c4e59ac4 c0448d8c c4e59ac4 c171f0c0 c0913590 c17c4300 c4e59ac4 c4e59b38 [ 4115.100000] 9aa0: c1704000 0ec38450 00000000 00001030 c0f67000 c0448eac c4e59ac4 c01b6d98 [ 4115.100000] 9ac0: 000000bc 00000000 00001030 00000000 00000000 00000000 00000000 c1704000 [ 4115.100000] 9ae0: 00000000 4f21ffb9 c1410800 c08dcbf8 c0fee000 00001030 00000004 00000761 [ 4115.100000] 9b00: 00018450 c1704000 c0913590 c0473c24 00001030 c4e59b38 c1704000 c0470f9c [ 4115.100000] 9b20: 0ec38450 00000000 00000000 c17c4300 c0913528 c0b42660 c17c4300 4f21ffb9 [ 4115.100000] 9b40: 4f21ffb9 00000000 c0fee000 c1410800 000006c5 c17c4300 00000000 c1704000 [ 4115.100000] 9b60: 00000761 c047177c 00001030 c02c363c c1630000 00001ec0 00000000 4f21ffb9 [ 4115.100000] 9b80: 00000761 4f21ffb9 c171f0c0 c1410800 00001030 00000000 00017450 000006c5 [ 4115.100000] 9ba0: c0fee000 c1704000 00000000 c047050c 00017450 00001030 00000000 4f21ffb9 [ 4115.100000] 9bc0: 4f21ffb9 c1630000 c1630000 00001030 000006c5 00017450 c17c4300 c4e59d10 [ 4115.100000] 9be0: 000006c5 c02c3050 00001030 00000000 c1630000 c0b3e460 c4e59c98 c1704000 [ 4115.100000] 9c00: c1630000 c02c6ddc 00001030 00000001 00001013 c02c84b0 c1630000 00017450 [ 4115.100000] 9c20: 00001030 00000001 c0be9e00 c0b0ec00 c17c4300 c0100bb8 00000001 4f21ffb9 [ 4115.100000] 9c40: c4e59ca8 c0100bb8 c27c8c20 c1704c50 00000360 29307c32 92b74a1c 4f21ffb9 [ 4115.100000] 9c60: 24da6e32 c1630000 00000000 c4e59d10 c16300e4 00000000 c1704000 00000000 [ 4115.100000] 9c80: c17c4300 c02c8874 20030013 00000801 00000003 c0be9e00 000042eb 2000460c [ 4115.100000] 9ca0: 00000000 000006c5 00017450 00001030 c17c4300 4f21ffb9 c1704000 c3fca920 [ 4115.100000] 9cc0: 00004f00 c11f7410 c0b3dc20 0000460c c17c4300 000042eb c1704000 c02b88c4 [ 4115.100000] 9ce0: 00000000 c0b57ef4 00000000 c0b3c32c 8af8af8b 00140cca 00352ac1 04f00000 [ 4115.100000] 9d00: c27c9000 c1630000 c17c4300 c3fca920 000042eb 2000460c c0b57ea4 4f21ffb9 [ 4115.100000] 9d20: c4e59d70 c3fca920 c17c4300 c1630000 c19ff600 0000460c 0000460b c11f7410 [ 4115.100000] 9d40: c11f7504 c02b9c90 c0b57ea4 c11f74f8 c3fca920 00000001 00000cc0 c01715d4 [ 4115.100000] 9d60: c4e59dc0 00000001 c17c4300 c3fca93c c11f74fc 4f21ffb9 000c0000 c3fca920 [ 4115.100000] 9d80: c17c4300 c4e59f28 c19ff600 c3fca920 00000000 00004620 c11f7504 c0170364 [ 4115.100000] 9da0: 00000000 00000cc0 c3fca920 0000460c c3fca920 c0171780 c4e59dc0 4f21ffb9 [ 4115.100000] 9dc0: 0034d869 c11f74f8 c4e59e8c c4e59f28 0000460c c3fca920 00000000 c017242c [ 4115.100000] 9de0: c198aabc c19ff600 c19ff650 c17c4300 00100008 c4e59f10 c0b02800 61c88647 [ 4115.100000] 9e00: c4e59f08 0000b000 c19ff600 c11f74f8 c19ff650 0000460c 00000000 00000000 [ 4115.100000] 9e20: c0b0b9cc 4f21ffb9 00007000 c11f7410 c4e59e90 c4e59f28 0000c000 c4e59f10 [ 4115.100000] 9e40: 00000000 00001000 00000000 c0175ed8 00001000 00000000 04620000 00000000 [ 4115.100000] 9e60: 10b072e6 c4e59f10 c19ff600 00000000 04f00000 00000000 00000000 c11f7410 [ 4115.100000] 9e80: c11f74f8 c17c4300 00000000 00020000 c3fca900 c19ff540 c4e59f08 c01c637c [ 4115.100000] 9ea0: 0000005e c0100bb8 c4e59f2c 00000000 ffffffe4 00000000 00020000 c19ff600 [ 4115.100000] 9ec0: 00000000 00004004 c4e59f78 4f21ffb9 00020000 00020000 c19ff600 04600000 [ 4115.100000] 9ee0: 00000000 c4e59f78 c17c4300 00020000 0000005e c01bdd74 00020000 4f21ffb9 [ 4115.100000] 9f00: b6ebe000 00020000 b6ebe000 00020000 00000000 0000c000 00014000 c4e59f08 [ 4115.100000] 9f20: 00000001 00000000 c19ff600 00000000 0460c000 00000000 00000000 00000000 [ 4115.100000] 9f40: 00000000 00004004 00000000 00000000 004660b4 4f21ffb9 b6ff5940 c19ff600 [ 4115.100000] 9f60: c19ff600 04600000 00000000 c17c4300 b6ebe000 c01be668 04600000 00000000 [ 4115.100000] 9f80: 00000003 4f21ffb9 004660b4 00020000 b6ff5940 00000003 c01002c4 c17c4300 [ 4115.100000] 9fa0: 00000003 c0100060 004660b4 00020000 00000003 b6ebe000 00020000 00000000 [ 4115.100000] 9fc0: 004660b4 00020000 b6ff5940 00000003 00000003 00020000 00020000 0000005e [ 4115.100000] 9fe0: 00000003 be907b58 b6f6d187 b6ef9c66 60070030 00000003 00000000 00000000 [ 4115.100000] atc_chain_complete from atc_advance_work+0x78/0x184 [ 4115.100000] atc_advance_work from atmel_nand_dma_transfer+0x114/0x23c [ 4115.100000] atmel_nand_dma_transfer from atmel_nand_data_in+0xd0/0x108 [ 4115.100000] atmel_nand_data_in from atmel_hsmc_exec_rw+0x34/0x3c [ 4115.100000] atmel_hsmc_exec_rw from nand_op_parser_exec_op+0x3b0/0x5c4 [ 4115.100000] nand_op_parser_exec_op from nand_read_data_op+0x1a4/0x27c [ 4115.100000] nand_read_data_op from atmel_nand_pmecc_read_pg.constprop.8+0x68/0xc4 [ 4115.100000] atmel_nand_pmecc_read_pg.constprop.8 from atmel_hsmc_nand_pmecc_read_page+0x1c/0x24 [ 4115.100000] atmel_hsmc_nand_pmecc_read_page from nand_read_oob+0x1bc/0x7c0 [ 4115.100000] nand_read_oob from mtd_read_oob+0x88/0x14c [ 4115.100000] mtd_read_oob from mtd_read+0x5c/0x80 [ 4115.100000] mtd_read from ubi_io_read+0xe4/0x36c [ 4115.100000] ubi_io_read from ubi_eba_read_leb+0xd0/0x4e8 [ 4115.100000] ubi_eba_read_leb from ubi_leb_read+0x94/0x100 [ 4115.100000] ubi_leb_read from ubifs_leb_read+0x2c/0x78 [ 4115.100000] ubifs_leb_read from fallible_read_node+0x84/0x27c [ 4115.100000] fallible_read_node from ubifs_tnc_locate+0xfc/0x1cc [ 4115.100000] ubifs_tnc_locate from do_readpage+0x19c/0x494 [ 4115.100000] do_readpage from ubifs_readpage+0x48/0x4a4 [ 4115.100000] ubifs_readpage from filemap_read_folio+0x44/0x1fc [ 4115.100000] filemap_read_folio from filemap_get_pages+0x4cc/0x790 [ 4115.100000] filemap_get_pages from filemap_read+0xcc/0x3bc [ 4115.100000] filemap_read from vfs_read+0x25c/0x2e4 [ 4115.100000] vfs_read from ksys_read+0xa0/0xd0 [ 4115.100000] ksys_read from ret_fast_syscall+0x0/0x54 [ 4115.100000] Exception stack(0xc4e59fa8 to 0xc4e59ff0) [ 4115.100000] 9fa0: 004660b4 00020000 00000003 b6ebe000 00020000 00000000 [ 4115.100000] 9fc0: 004660b4 00020000 b6ff5940 00000003 00000003 00020000 00020000 0000005e [ 4115.100000] 9fe0: 00000003 be907b58 b6f6d187 b6ef9c66 [ 4115.100000] Code: ebf78ba1 e3a03000 e5c43068 eaffffc3 (e7f001f2) [ 4115.100000] ---[ end trace 0000000000000000 ]---
