On Wed, Apr 08, 2020 at 02:29:51PM +0100, John Garry wrote: > On 08/04/2020 14:10, Daniel Wagner wrote: > ok, but to really test this you need to ensure that all the cpus for a > managed interrupt affinity mask are offlined together for some period of > time greater than the IO timeout. Otherwise the hw queue's managed interrupt > would not be shut down, and you're not verifying that the queues are fully > drained. Not sure if I understand you correctly: Are you saying that the IRQ related resources are not freed/moved from the offlining CPU? > > > Will the fio processes migrate back onto cpus which have been onlined again? > > > > Hmm, good question. I've tried to assign them to a specific CPU via > > --cpus_allowed_policy=split and --cpus_allowed. > > > > fio --rw=randwrite --name=test --size=50M --iodepth=32 --direct=1 \ > > --bs=4k --numjobs=40 --time_based --runtime=1h --ioengine=libaio \ > > --group_reporting --cpus_allowed_policy=split --cpus_allowed=0-40 > > > > Though I haven't verified what happens when the CPU get's back online. > > Maybe this will work since you're offlining patterns of cpus and the fio > processes have to migrate somewhere. But see above. At least after the initial setup a fio thread will be migrated away from the offlining CPU. A quick test shows, that the affinity mask for a fio thread will be cleared when the CPU goes offline. There seems to be a discussion going on about the cpu hotplug and the affinity mask: https://lore.kernel.org/lkml/1251528473.590671.1579196495905.JavaMail.zimbra@xxxxxxxxxxxx TL;DR: it can be scheduled back if affinity is tweaked via e.g. taskset, it won't if it's via cpusets Thanks, Daniel
