On 8/10/2022 3:45 PM, Don.Brace@xxxxxxxxxxxxx wrote:
I have been reading the kernel documentation about using a dmengine provider/client to see if the md driver will utilize a DMA engine when doing XOR and Crypto operations. I notice that the drivers/md/raid5.c calls async_xor_offs() which is found in crypto/async_tx/async_xor.c and it is calling async_tx_find_channel(). So, I think the answer is yes if a DMA engine is enabled in the kernel. Is this correct? I did some tracing while doing I/O to my raid5 with crypto enabled and see the above functions called but unsure of how data flows through each driver and if I am even using a DMA offload.
What platform are you running on? There are some ARM SOC DMA engines that support XOR such as the Marvell chip (mv_xor) as I recall. Intel Xeon platforms supported that long while ago. But that has been removed since Skylake. If you do a grep in drivers/dma/ for DMA_XOR where the driver calls dma_cap_set(DMA_XOR, ...) you can see which drivers supports RAID5 offload.
I have the following drivers loaded: lsmod | grep raid raid456 188416 2 async_raid6_recov 24576 1 raid456 async_memcpy 20480 2 raid456,async_raid6_recov async_pq 20480 2 raid456,async_raid6_recov async_xor 20480 3 async_pq,raid456,async_raid6_recov async_tx 20480 5 async_pq,async_memcpy,async_xor,raid456,async_raid6_recov raid6_pq 122880 3 async_pq,raid456,async_raid6_recov libcrc32c 16384 5 nf_conntrack,nf_nat,nf_tables,xfs,raid456 Is there a diagram somewhere that provides any details? I made the raid5 with crypto using mdadm --create /dev/md/raid5 --force --assume-clean --verbose --level=5 --chunk=512K --metadata=1 --data-offset=2048s --raid-devices=5 /dev/mapper/mpathb /dev/mapper/mpathc /dev/mapper/mpathd /dev/mapper/mpathe /dev/mapper/mpathl cryptsetup -v luksFormat /dev/md/raid5Crypto cryptsetup luksOpen /dev/md/raid5Crypto testCrypto mkfs.ext4 /dev/mapper/testCrypto
