On Fri, Aug 06, 2021 at 07:27:13AM +0000, Kasireddy, Vivek wrote: > Hi Daniel, > > > > > > >>> The solution: > > > > > >>> - To ensure full framerate, the Guest compositor has to start it's repaint cycle > > > > (including > > > > > >>> the 9 ms wait) when the Host compositor sends the frame callback event to its > > > > clients. > > > > > >>> In order for this to happen, the dma-fence that the Guest KMS waits on -- before > > > > sending > > > > > >>> pageflip completion -- cannot be tied to a wl_buffer.release event. This means > > that, > > > > the > > > > > >>> Guest compositor has to be forced to use a new buffer for its next repaint cycle > > > > when it > > > > > >>> gets a pageflip completion. > > > > > >> > > > > > >> Is that really the only solution? > > > > > > [Kasireddy, Vivek] There are a few others I mentioned here: > > > > > > https://gitlab.freedesktop.org/wayland/weston/-/issues/514#note_986572 > > > > > > But I think none of them are as compelling as this one. > > > > > > > > > > > >> > > > > > >> If we fix the event timestamps so that both guest and host use the same > > > > > >> timestamp, but then the guest starts 5ms (or something like that) earlier, > > > > > >> then things should work too? I.e. > > > > > >> - host compositor starts at (previous_frametime + 9ms) > > > > > >> - guest compositor starts at (previous_frametime + 4ms) > > > > > >> > > > > > >> Ofc this only works if the frametimes we hand out to both match _exactly_ > > > > > >> and are as high-precision as the ones on the host side. Which for many gpu > > > > > >> drivers at least is the case, and all the ones you care about for sure :-) > > > > > >> > > > > > >> But if the frametimes the guest receives are the no_vblank fake ones, then > > > > > >> they'll be all over the place and this carefully tuned low-latency redraw > > > > > >> loop falls apart. Aside fromm the fact that without tuning the guests to > > > > > >> be earlier than the hosts, you're guaranteed to miss every frame (except > > > > > >> when the timing wobbliness in the guest is big enough by chance to make > > > > > >> the deadline on the oddball frame). > > > > > > [Kasireddy, Vivek] The Guest and Host use different event timestamps as we don't > > > > > > share these between the Guest and the Host. It does not seem to be causing any > > other > > > > > > problems so far but we did try the experiment you mentioned (i.e., adjusting the > > > > delays) > > > > > > and it works. However, this patch series is meant to fix the issue without having to > > > > tweak > > > > > > anything (delays) because we can't do this for every compositor out there. > > > > > > > > > > Maybe there could be a mechanism which allows the compositor in the guest to > > > > automatically adjust its repaint cycle as needed. > > > > > > > > > > This might even be possible without requiring changes in each compositor, by > > adjusting > > > > the vertical blank periods in the guest to be aligned with the host compositor repaint > > > > cycles. Not sure about that though. > > > > > > > > > > Even if not, both this series or making it possible to queue multiple flips require > > > > corresponding changes in each compositor as well to have any effect. > > > > > > > > Yeah from all the discussions and tests done it sounds even with a > > > > deeper queue we have big coordination issues between the guest and > > > > host compositor (like the example that the guest is now rendering at > > > > 90fps instead of 60fps like the host). > > > [Kasireddy, Vivek] Oh, I think you are referring to my reply to Gerd. That 90 FPS vs > > > 60 FPS problem is a completely different issue that is associated with Qemu GTK UI > > > backend. With the GTK backend -- and also with SDL backend -- we Blit the Guest > > > scanout FB onto one of the backbuffers managed by EGL. > > > > > > I am trying to add a new Qemu Wayland UI backend so that we can eliminate that Blit > > > and thereby have a truly zero-copy solution. And, this is there I am running into the > > > halved frame-rate issue -- the current problem. > > > > Yes, that's what I referenced. But I disagree that it's a different > > problem. The underlying problem in both cases is that the guest and host > > compositor free-wheel instead of rendering in sync. It's just that > > depending upon how exactly the flip completion event on the gues side > > plays out you either get guest rendering that's faster than the host-side > > 60fps, or guest rendering that's much slower than the host-side 60fps. > [Kasireddy, Vivek] That used to be the case before we added a synchronization > mechanism to the GTK UI backend that uses a sync file. After adding this > and making the Guest wait until this sync file fd on the Host is signaled, we > consistently get 60 FPS because the flip completion event for the Guest is > directly tied to the signaling of the sync file in this particular case (GTK UI). > > > > > The fundamental problem in both cases is that they don't run in lockstep. > > If you fix that, through fixing the timestamp and even reporting most > > likely, you should be able to fix both bugs. > [Kasireddy, Vivek] GTK UI is an EGL based solution that Blits the Guest scanout > FB onto one of the backbuffers managed by EGL. Wayland UI is a zero-copy > solution that just wraps the dmabuf associated with Guest scanout FB in a > wl_buffer and submits it directly to the Host compositor. These backends are > completely independent of each other and cannot be active at the same time. > In other words, we cannot have zero-copy and Blit based solutions running > parallelly. And, this issue is only relevant for Wayland UI backend and has > nothing to do with GTK UI. > > > > > > > Hence my gut feeling reaction that first we need to get these two > > > > compositors aligned in their timings, which propobably needs > > > > consistent vblank periods/timestamps across them (plus/minux > > > > guest/host clocksource fun ofc). Without this any of the next steps > > > > will simply not work because there's too much jitter by the time the > > > > guest compositor gets the flip completion events. > > > [Kasireddy, Vivek] Timings are not a problem and do not significantly > > > affect the repaint cycles from what I have seen so far. > > > > > > > > > > > Once we have solid events I think we should look into statically > > > > tuning guest/host compositor deadlines (like you've suggested in a > > > > bunch of places) to consisently make that deadline and hit 60 fps. > > > > With that we can then look into tuning this automatically and what to > > > > do when e.g. switching between copying and zero-copy on the host side > > > > (which might be needed in some cases) and how to handle all that. > > > [Kasireddy, Vivek] As I confirm here: https://gitlab.freedesktop.org/wayland/weston/- > > /issues/514#note_984065 > > > tweaking the deadlines works (i.e., we get 60 FPS) as we expect. However, > > > I feel that this zero-copy solution I am trying to create should be independent > > > of compositors' deadlines, delays or other scheduling parameters. > > > > That's not how compositors work nowadays. Your problem is that you don't > > have the guest/host compositor in sync. zero-copy only changes the timing, > > so it changes things from "rendering way too many frames" to "rendering > > way too few frames". > > > > We need to fix the timing/sync issue here first, not paper over it with > > hacks. > [Kasireddy, Vivek] What I really meant is that the zero-copy solution should be > independent of the scheduling policies to ensure that it works with all compositors. > IIUC, Weston for example uses the vblank/pageflip completion timestamp, the > configurable repaint-window value, refresh-rate, etc to determine when to start > its next repaint -- if there is any damage: > timespec_add_nsec(&output->next_repaint, stamp, refresh_nsec); > timespec_add_msec(&output->next_repaint, &output->next_repaint, -compositor->repaint_msec); > > And, in the case of VKMS, since there is no real hardware, the timestamp is always: > now = ktime_get(); > send_vblank_event(dev, e, seq, now); vkms has been fixed since a while to fake high-precision timestamps like from a real display. > When you say that the Guest/Host compositor need to stay in sync, are you > suggesting that we need to ensure that the vblank timestamp on the Host > needs to be shared and be the same on the Guest and a vblank/pageflip > completion for the Guest needs to be sent at exactly the same time it is sent > on the Host? If yes, I'd say that we do send the pageflip completion to Guest > around the same time a vblank is generated on the Host but it does not help > because the Guest compositor would only have 9 ms to submit a new frame > and if the Host is running Mutter, the Guest would only have 2 ms. > (https://gitlab.freedesktop.org/wayland/weston/-/issues/514#note_984341) Not at the same time, but the same timestamp. And yes there is some fun there, which is I think the fundamental issue. Or at least some of the compositor experts seem to think so, and it makes sense to me. > > > > Only, and I really mean only, when that shows that it's simply impossible > > to hit 60fps with zero-copy and the guest/host fully aligned should we > > look into making the overall pipeline deeper. > [Kasireddy, Vivek] From all the experiments conducted so far and given the > discussion associated with https://gitlab.freedesktop.org/wayland/weston/-/issues/514 > I think we have already established that in order for a zero-copy solution to work > reliably, the Guest compositor needs to start its repaint cycle when the Host > compositor sends a frame callback event to its clients. > > > > > > > Only when that all shows that we just can't hit 60fps consistently and > > > > really need 3 buffers in flight should we look at deeper kms queues. > > > > And then we really need to implement them properly and not with a > > > > mismatch between drm_event an out-fence signalling. These quick hacks > > > > are good for experiments, but there's a pile of other things we need > > > > to do first. At least that's how I understand the problem here right > > > > now. > > > [Kasireddy, Vivek] Experiments done so far indicate that we can hit 59 FPS consistently > > > -- in a zero-copy way independent of compositors' delays/deadlines -- with this > > > patch series + the Weston MR I linked in the cover letter. The main reason why this > > > works is because we relax the assumption that when the Guest compositor gets a > > > pageflip completion event that it could reuse the old FB it submitted in the previous > > > atomic flip and instead force it to use a new one. And, we send the pageflip completion > > > event to the Guest when the Host compositor sends a frame callback event. Lastly, > > > we use the (deferred) out_fence as just a mechanism to tell the Guest compositor when > > > it can release references on old FBs so that they can be reused again. > > > > > > With that being said, the only question is how can we accomplish the above in an > > upstream > > > acceptable way without regressing anything particularly on bare-metal. Its not clear if > > just > > > increasing the queue depth would work or not but I think the Guest compositor has to be > > told > > > when it can start its repaint cycle and when it can assume the old FB is no longer in use. > > > On bare-metal -- and also with VKMS as of today -- a pageflip completion indicates > > both. > > > In other words, Vblank event is the same as Flip done, which makes sense on bare-metal. > > > But if we were to have two events at-least for VKMS: vblank to indicate to Guest to start > > > repaint and flip_done to indicate to drop references on old FBs, I think this problem can > > > be solved even without increasing the queue depth. Can this be acceptable? > > > > That's just another flavour of your "increase queue depth without > > increasing the atomic queue depth" approach. I still think the underlying > > fundamental issue is a timing confusion, and the fact that adjusting the > > timings fixes things too kinda proves that. So we need to fix that in a > > clean way, not by shuffling things around semi-randomly until the specific > > config we tests works. > [Kasireddy, Vivek] This issue is not due to a timing or timestamp mismatch. We > have carefully instrumented both the Host and Guest compositors and measured > the latencies at each step. The relevant debug data only points to the scheduling > policy -- of both Host and Guest compositors -- playing a role in Guest rendering > at 30 FPS. Hm but that essentially means that the events your passing around have an even more ad-hoc implementation specific meaning: Essentially it's the kick-off for the guest's repaint loop? That sounds even worse for a kms uapi extension. > > Iow I think we need a solution here which both slows down the 90fps to > > 60fps for the blit case, and the 30fps speed up to 60fps for the zerocopy > > case. Because the host might need to switch transparently between blt and > > zerocopy for various reasons. > [Kasireddy, Vivek] As I mentioned above, the Host (Qemu) cannot switch UI > backends at runtime. In other words, with GTK UI backend, it is always Blit > whereas Wayland UI backend is always zero-copy. Hm ok, that at least makes things somewhat simpler. Another thing that I just realized: What happens when the host changes screen resolution and especially refresh rate? -Daniel > > Thanks, > Vivek > > > -Daniel > > > > > Thanks, > > > Vivek > > > > > > > > Cheers, Daniel > > > > > > > > > > > > > > > > > > > -- > > > > > Earthling Michel Dänzer | https://redhat.com > > > > > Libre software enthusiast | Mesa and X developer > > > > > > > > > > > > > > > > -- > > > > Daniel Vetter > > > > Software Engineer, Intel Corporation > > > > http://blog.ffwll.ch > > > > -- > > Daniel Vetter > > Software Engineer, Intel Corporation > > http://blog.ffwll.ch -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
