> On Mon, May 09, 2022 at 01:17:00PM -0400, Joel Fernandes wrote: > > On Sun, May 8, 2022 at 11:37 PM Paul E. McKenney <paulmck@xxxxxxxxxx> wrote: > > > > > > On Sun, May 08, 2022 at 08:17:49PM -0400, Joel Fernandes wrote: > > > > On Sun, May 8, 2022 at 5:32 PM Paul E. McKenney <paulmck@xxxxxxxxxx> wrote: > > [...] > > > > > > Also, I think it is wrong to assume that a certain kind of system will > > > > > > always have a certain number of callbacks to process at a time. That > > > > > > seems prone to poor design due to assumptions which may not always be > > > > > > true. > > > > > > > > > > Who was assuming that? Uladzislau was measuring rather than assuming, > > > > > if that was what you were getting at. Or if you are thinking about > > > > > things like qhimark, your point is exactly why there is both a default > > > > > (which has worked quite well for a very long time) and the ability to > > > > > adjust based on the needs of your specific system. > > > > > > > > I was merely saying that based on measurements make assumptions, but > > > > in the real world the assumption may not be true, then everything > > > > falls apart. Instead I feel, callback threads should be RT only if 1. > > > > As you mentioned, the time based thing. 2. If the CB list is long and > > > > there's lot of processing. But instead, if it is made a CONFIG option, > > > > then that forces a fixed behavior which may fall apart in the real > > > > world. I think adding more CONFIGs and special cases is more complex > > > > but that's my opinion. > > > > > > Again, exactly what problem are you trying to solve? > > > > > > From what I can see, Uladzislau's issue can be addressed by statically > > > setting the rcuo kthreads to SCHED_OTHER at boot time. The discussion > > > is on exactly how RCU is to be informed of this, at kernel build time. > > > > > > > > > Can we not have 2 sets of RCU offload threads, one which operate at RT > > > > > > and only process few callbacks at a time, while another which is the > > > > > > lower priority CFS offload thread - executes whenever there is a lot > > > > > > of CBs pending? Just a thought. > > > > > > > > > > How about if we start by solving the problems we know that we have? > > > > > > > > I don't know why you would say that, because we are talking about > > > > solving the specific problem Vlad's patch addresses, not random > > > > problems. Which is that, Android wants to run expedited GPs, but when > > > > the callback list is large, the RT nocb thread can starve other > > > > things. Did I misunderstand the patch? If so, sorry about that but > > > > that's what my email was discussing. i.e. running of CBs in RT > > > > threads. I suck at writing well as I clearly miscommunicated. > > > > > > OK. > > > > > > Why do you believe that this needs anything other than small adjustments > > > the defaults of existing Kconfig options? Or am I completely missing > > > the point of your proposal? > > > > > > > > > Otherwise, I feel like we might be again proliferating CONFIG options > > > > > > and increasing burden on the user to get it the CONFIG right. > > > > > > > > > > I bet that we will solve this without adding any new Kconfig options. > > > > > And I bet that the burden is at worst on the device designer, not on > > > > > the user. Plus it is entirely possible that there might be a way to > > > > > automatically configure things to handle what we know about today, > > > > > again without adding Kconfig options. > > > > > > > > Yes, agreed. > > > > > > If I change my last sentence to read as follows, are we still in > > > agreement? > > > > > > Plus it is entirely possible that there might be a way to > > > automatically configure things to handle what we know about today, > > > again without adding Kconfig options and without changing runtime > > > code beyond that covered by Uladzislau's patch. > > > > Yes, actually the automatic configuration of things is what I meant, > > that's the "problem" I was referring to, where the system does the > > right thing for a broader range of systems, without requiring the > > users to find RCU issues and hand-tune them (that requires said users > > to have tracing and debugging skills and get lucky finding a problem). > > To be fair, I did not propose any solutions to such problems either, > > it is just some ideas. I don't like knobs too much and I don't trust > > users or system designers to get them right most of the time. > > > > In that sense, I don't think making rcuo threads run as RT or not > > (which this patch does) is really fixing the problems. In one case, > > you might have priority inversion, in another case you might cause > > starvation. Probably what is needed is best of both worlds. That said, > > I don't have better solutions right now than what I mentioned, which > > is to assign priorities to the callbacks themselves and run them in > > threads of different priorities. > > > > For the record, I am not against the patch or anything like that (and > > even if I was, I am not sure that it matters for merging :P) > > Fair enough! > > And for the record at this end, I would not be surprised if in 2032 > RCU offloaded callback invocation has sophisticated dynamic tuning of > priorities and much else besides. But one step at a time! ;-) > hh... It is hard to comment because i am a bit lost in this big conversation :) What i have got so far. Joel does not like adding extra *_CONFIG options, actually me too since it becomes more complicated thus it requires more specific attention from users. I prefer to make the code common but it is not possible sometimes to make it common, because we have different kind of kernels and workloads. >From the other hand the patch splits the BOOSTING logic into two peaces because driving the grace periods kthreads in RT priority is not a big issue because their run-times are short. Whereas running the "kthreads-callbacks" in the RT context can be long so we end up in throttled situation for other workloads. I see that Paul would like to keep it for CONFIG_PREEMPT_RT, because it was mainly designed for that kind of kernels. So we can align with Alison patch and her decision, so i do not see any issues. So far RT folk seems does not mind in having "callback-kthreads" as SCHED_FIFO :) Do you agree with start from keeping it ON for CONFIG_PREEMPT_RT conf. by default and OFF for other cases? -- Uladzislau Rezki
