On Fri, 16 Mar 2018 09:52:58 -0400 John Pittman <jpittman@xxxxxxxxxx> wrote: > Through numerous and consistent customer cases, it has been noted > that on systems with above average or high load, md devices backed > with rotational media cause a significant, system-wide I/O performance > impact during resync. This includes, but is not limited to, the > installation environment when root is on a md device. For all intents > and purposes, due to drastically increased seek operations, this > behavior is completely warranted and expected. However, it does cause > resync operations to only be truly feasible on low load systems or > during downtime. As devices grow larger, resyncs are taking longer, > requiring feasibility to extend into production uptime. So, taking > this into account, for rotational devices, 200000 is no longer a > reasonable limit. It's been found that when these performance issues > are encountered, in virtually all cases, the issue can be completely > resolved by setting a sync_speed_max value somewhere in between 50000 > and 100000, the lower it's set, the better the performance gets, as > expected. It has been reported[1] that this issue appeared mostly after the patch[2], "md: allow resync to go faster when there is competing IO." [1] https://www.spinics.net/lists/raid/msg51363.html [2] https://github.com/torvalds/linux/commit/ac8fa4196d205ac8fff3f8932bddbad4f16e4110 ...and that reverting it helps the concurrent seek performance immensely. Could you try reverting that patch (it still reverts cleanly on current 4.14.27) and retest those findings? Perhaps that's where the issue should be fixed, rather than just worked around partially via lowering the speed limit. > Avoid these performance hits by persistently setting rotational devices, > via a udev rule, to a more reasonable value of 100000. And in any case IMHO an udev rule feels to be a wrong place to fix this, if it's decided that a different value than 200000 should be used, then perhaps the default in the kernel itself can be changed? -- With respect, Roman -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
