On 12.11.21 16:45, Moessbauer, Felix (T RDA IOT SES-DE) wrote: > Hi Sebastian, > >> -----Original Message----- >> From: Sebastian Andrzej Siewior <bigeasy@xxxxxxxxxxxxx> >> Sent: Friday, November 12, 2021 4:37 PM >> To: Moessbauer, Felix (T RDA IOT SES-DE) <felix.moessbauer@xxxxxxxxxxx>; >> cgroups@xxxxxxxxxxxxxxx >> Cc: linux-rt-users@xxxxxxxxxxxxxxx; Schild, Henning (T RDA IOT SES-DE) >> <henning.schild@xxxxxxxxxxx>; Kiszka, Jan (T RDA IOT) >> <jan.kiszka@xxxxxxxxxxx>; Schmidt, Adriaan (T RDA IOT SES-DE) >> <adriaan.schmidt@xxxxxxxxxxx>; Frederic Weisbecker <frederic@xxxxxxxxxx> >> Subject: Re: Questions about replacing isolcpus by cgroup-v2 >> >> On 2021-11-04 17:29:08 [+0000], Moessbauer, Felix wrote: >>> Dear subscribers, >> Hi, >> >> I Cced cgroups@vger since thus question fits there better. >> I Cced Frederic in case he has come clues regarding isolcpus and cgroups. > > Indeed. Thanks! > >> >>> we are currently evaluating how to rework realtime tuning to use cgroup-v2 >> cpusets instead of the isolcpus kernel parameter. >>> Our use-case are realtime applications with rt and non-rt threads. Hereby, the >> non-rt thread might create additional non-rt threads: >>> >>> Example (RT CPU=1, 4 CPUs): >>> - Non-RT Thread (A) with default affinity 0xD (1101b) >>> - RT Thread (B) with Affinity 0x2 (0010b, via set_affinity) >>> >>> When using pure isolcpus and cgroup-v1, just setting isolcpus=1 perfectly >> works: >>> Thread A gets affinity 0xD, Thread B gets 0x2 and additional threads get a >> default affinity of 0xD. >>> By that, independent of the threads' priorities, we can ensure that nothing is >> scheduled on our RT cpu (except from kernel threads, etc...). >>> >>> During this journey, we discovered the following: >>> >>> Using cgroup-v2 cpusets and isolcpus together seems to be incompatible: >>> When activating the cpuset controller on a cgroup (for the first time), all >> default CPU affinities are reset. >>> By that, also the default affinity is set to 0xFFFF..., while with isolcpus we >> expect it to be (0xFFFF - isolcpus). >>> This breaks the example from above, as now the non-RT thread can also be >> scheduled on the RT CPU. >>> >>> When only using cgroup-v2, we can isolate our RT process by placing it in a >> cgroup with CPUs=0,1 and remove CPU=1 from all other cgroups. >>> However, we do not know of a strategy to set a default affinity: >>> Given the example above, we have no way to ensure that newly created >> threads are born with an affinity of just 0x2 (without changing the application). >>> >>> Finally, isolcpus itself is deprecated since kernel 5.4. >> >> Where is this the deprecation of isolcpus announced/ written? > > https://www.kernel.org/doc/html/latest/admin-guide/kernel-parameters.html > isolcpus= [KNL,SMP,ISOL] Isolate a given set of CPUs from disturbance. > [Deprecated - use cpusets instead] > Format: [flag-list,]<cpu-list> > That was Frederic himself via b0d40d2b22fe - but already for 4.15... Jan -- Siemens AG, T RDA IOT Corporate Competence Center Embedded Linux
