On Thu, Mar 26, 2020 at 03:46:23PM +0100, Vitaly Kuznetsov wrote: > "Andrea Parri (Microsoft)" <parri.andrea@xxxxxxxxx> writes: > > > VMBus version 4.1 and later support the CHANNELMSG_MODIFYCHANNEL(22) > > message type which can be used to request Hyper-V to change the vCPU > > that a channel will interrupt. > > > > Introduce the CHANNELMSG_MODIFYCHANNEL message type, and define the > > vmbus_send_modifychannel() function to send CHANNELMSG_MODIFYCHANNEL > > requests to the host via a hypercall. The function is then used to > > define a sysfs "store" operation, which allows to change the (v)CPU > > the channel will interrupt by using the sysfs interface. The feature > > can be used for load balancing or other purposes. > > > > One interesting catch here is that Hyper-V can *not* currently ACK > > CHANNELMSG_MODIFYCHANNEL messages with the promise that (after the ACK > > is sent) the channel won't send any more interrupts to the "old" CPU. > > > > The peculiarity of the CHANNELMSG_MODIFYCHANNEL messages is problematic > > if the user want to take a CPU offline, since we don't want to take a > > CPU offline (and, potentially, "lose" channel interrupts on such CPU) > > if the host is still processing a CHANNELMSG_MODIFYCHANNEL message > > associated to that CPU. > > > > It is worth mentioning, however, that we have been unable to observe > > the above mentioned "race": in all our tests, CHANNELMSG_MODIFYCHANNEL > > requests appeared *as if* they were processed synchronously by the > > host. > > Hyper-V engineers never want to make our lifes easier :-) Haha. I'd say more exciting! ;-) ;-) > I can only think of a 'lazy' approach to setting channel CPU affinity: > we actually re-assign it to the target CPU when we receive first > interrupt there - but it's not very different from re-assigning it there > but still handling interrupts on the old CPU like you do. Interesting! I'm wondering whether it'd make sense to use a similar approach to (lazily) "unblock" the "old" CPUs; let me think more... > One more thing: it was already discussed several times but we never get > to it. I think this question was even raised on Michael's latest > 'Hyper-V on ARM' submission. What about implmenting a Hyper-V specific > IRQ chip which would now support setting CPU affinity? The greatest > benefit is that we don't need new tools to do e.g. load balancing, > irqbalance will just work. Thank you for the suggestions; TBH, I haven't considered such approach so far (and I'd need more time to come up with an informed comment...) OTOH, I had some initial investigations about the current (in-kernel) balancing scheme/init_vp_index() and possible improvements/extensions there... Hopefully another, follow-up series to come soon! Thanks, Andrea
