I've been testing simple raid5 consisting of 4 ssds (all handling
post-trim reads as 0s safely, as reported by hdaprm -I: "Deterministic
read ZEROs after TRIM") - with 4.19.13 and 4.20.0 as of now (so decently
fresh kernels).
The drives are itself 2T (1.9T after overprovisioning with hpa), so
initial tests were done with 5.7T array. but that was completely
unusable in practice, the moment discards were attempted.
Then I created tiny array from 32gb long chunks (pre-discarded manually):
# echo Y >/sys/module/raid456/parameters/devices_handle_discard_safely
# mdadm -C /dev/md/test32 -e1.1 -l5 -n4 -z32G --assume-clean -n test32
/dev/sd{a,b,c,d}1
Then I did simple mkfs.ext4 /dev/md/test32
It took over 11 minutes to finish (of which nearly everything was
accounted for "Discarding device blocks" phase). Extrapolating to
full size array that would be around 7 hours.
The drives alone (full 1.9T size) handle mkfs.ext4 with full discard
phase in 1 minute or so. Quick test with raid10 works fine as well
(albeit a bit slower than I expected - a bit under 6 minutes; full
fstrim 37min - but that's perfectly fine for daily/weekly fstrim).
Is raid5's trim implementation expected to be this slow ?
