On 04.11.21 16:54, Eric Farman wrote: > On Thu, 2021-11-04 at 15:59 +0100, David Hildenbrand wrote: >>>> For example, we don't care about concurrent SIGP SENSE. We only >>>> care >>>> about "lightweight" SIGP orders with concurrent "heavy weight" >>>> SIGP >>>> orders. >>> >>> I very much care about concurrent SIGP SENSE (a "lightweight" order >>> handled in-kernel) and how that interacts with the "heavy weight" >>> SIGP >>> orders (handled in userspace). SIGP SENSE might return CC0 >>> (accepted) >>> if a vcpu is operating normally, or CC1 (status stored) with status >>> bits indicating an external call is pending and/or the vcpu is >>> stopped. >>> This means that the actual response will depend on whether >>> userspace >>> has picked up the sigp order and processed it or not. Giving CC0 >>> when >>> userspace is actively processing a SIGP STOP/STOP AND STORE STATUS >>> would be misleading for the SIGP SENSE. (Did the STOP order get >>> lost? >>> Failed? Not yet dispatched? Blocked?) >> >> But that would only visible when concurrently SIGP STOP'ing from one >> VCPU and SIGP SENSE'ing from another VCPU. But in that case, there >> are >> already no guarantees, because it's inherently racy: >> >> VCPU #2: SIGP STOP #3 >> VCPU #1: SIGP SENSE #3 >> > > Is it inherently racy? QEMU has a global "one SIGP at a time, > regardless of vcpu count" mechanism, so that it gets serialized at that > level. POPS says an order is rejected (BUSY) if the "access path to a > cpu is processing another order", and I would imagine that KVM is > acting as that access path to the vcpu. The deliniation between > kernelspace and userspace should be uninteresting on whether parallel > orders are serialized (in QEMU via USER_SIGP) or not (!USER_SIGP or > "lightweight" orders). There is no real way for a guest to enforce the execution order of VCPU #2: SIGP STOP #3 VCPU #1: SIGP SENSE #3 or VCPU #1: SIGP SENSE #3 VCPU #2: SIGP STOP #3 without additional synchronization. There could be random delays in the instruction execution at any point in time. So the SENSE on #2 might observe "stopped" "not stopped" or "busy" randomly, because it's inherently racy. Of course, one could implement some synchronization on top: VCPU #2: SIGP STOP #3 # VCPU #2 instructs #1 to SIGP SENSE #2 VCPU #1: SIGP SENSE #3 # VCPU #2 waits for SIGP SENSE #2 result from #1 VCPU #2: SIGP SENSE #3 Then, we have to make sure that it cannot happen that #1 observes "not busy" and #2 observes "busy". But, to implement something like that, #2 has to execute additional instructions to perform the synchronization. So after SIGP STOP returns on #2 and #2 was able to execute new instructions, we have to make sure that SIGP SENSE of #3 returns "busy" on all VCPUs until #3 finished the SIGP STOP. > >> There is no guarantee who ends up first >> a) In the kernel >> b) On the final destination (SENSE -> kernel; STOP -> QEMU) >> >> They could be rescheduled/delayed in various ways. >> >> >> The important part is that orders from the *same* CPU are properly >> handled, right? >> >> VCPU #1: SIGP STOP #3 >> VCPU #1: SIGP SENSE #3 >> >> SENSE must return BUSY in case the STOP was not successful yet, >> correct? > > It's not a matter of whether STOP is/not successful. If the vcpu is Right, I meant "accepted but not fully processed yet". > actively processing a STOP, then the SENSE gets a BUSY. But there's no > code today to do that for the SENSE, which is of course why I'm here. > :) Right, and the only problematic SIGP orders are really SIGP STOP*, because these are the only ones that will get processed asynchronously -- the sending VCPU can return and execute new instructions without the SIGP STOP order being fully processed. -- Thanks, David / dhildenb
