Hi, As some of you might be aware Collabora has been working, for some time, on a new compositor/display infrastructure which we believe is ready for a technology preview for those who want to give it a try. With this announcement we would like to write a few words about what has changed, how to set it up, and what are the next potential action items to look into. Note this is on-going work, and might suffer some intermediary breakage. Some of the details are still a bit fuzzy/in discussion and might suffer additional changes. # Terminology, current architecture and changes Wayland, as most of you know, is only a protocol, with some primitives to allow marshalling data between a client and a server and with some higher protocol specification on top of those primitives defined with the help of a XML file. The protocol itself used to (and still does, but it has since been deprecated) have window management functionality (wl_shell) but as those requirements have been largely revised it was decided to migrate those to its own protocol extension called xdg-shell. Weston is a server that implements the Wayland protocol. It is focused on correctness and efficient HW usage. Weston is built on top of libweston which has a front-facing API and an internal API to abstract the HW and allow the compositor developer focus on providing the required functionality as they deems necessary. Another compositor built on top of libweston is agl-compositor, which is the main subject of this email. Besides libweston, weston architecture was designed to allow plug-ins to be loaded at run-time which can implement even more functionality. But more on that later, a few paragraphs below. xdg-shell is currently de facto protocol for handling all window management related functionality. In order to avoid polluting the wayland protocol namespace, the XDG extension, together with some other useful protocols, like the linux-dmabuf one, are all developed under the wayland-protocols umbrella. The whole purpose of wayland-protocols is to enhance the Wayland protocol with new functionality and bring new extensions entirely. Compositors are free to implement, modify, enhance, and add new extensions to wayland-protocols but they need to do so in consensus. Besides the core wayland protocol and extended functionality from wayland-protocols, a compositor can provide additional protocol extension specifications (obviously specific to that compositor). Another window management extension instead of xdg-shell, that abides to the GENIVI specification (Infotainment), is the ivi-shell: a Weston module that implements ivi-application protocol specification. Together with this ivi-shell, further additional functionality has also been brought in with wayland-ivi-extension (which allows, among other things, to specify the position of windows). The ivi-shell plugin has been designed to allow loading of additional modules, called controllers, which each vendor can adapt and modify on their own. Weston provides hmi-controller (as an example), and wayland-ivi-extension provides ivi-controller. Graphical widget toolkits like Qt will abstract the underlying window management in use, and can at run-time, switch between ivi-shell and xdg-shell. In the same manner, the windowmanager currently found in AGL is responsible for abstracting the ivi-shell interaction and it used for managing and controlling the Layers, Screens and Surfaces from ivi-shell. Last in our architecture is the graphical user interface called homescreen and is implemented with Qt. It spawns applications and switches to/activates/displays them. It does that by using a library (libhomescreen) which is another abstraction built on top of a binder, called agl-service-homescreen. The binder talks with afm-system-daemon to start the application (if not already started) prior to switching/activating it by using windowmanager/libwindowmanager and the service with the same. The binder can also be used to switch/activate other applications and it is also in use by the launcher application. With the introduction of agl-compositor we aim to simplify the design and trim down this chain of abstractions while retaining the ability to modify and adapt for each vendor's needs and requirements. The approach we have taken is to create a new, simpler, but powerful compositor and to use xdg-shell instead of relying further on ivi-shell and its convoluted design. The compositor is built on top of libweston, and has been designed to contain only the bare minimum to bring up the display and the main event loop. As all the heavy lifting is done inside libweston, vendors are encouraged to study, adapt, and modify the compositor as they see fit. The xdg-shell server side implementation is hidden behind libweston-desktop library which agl-compositor is a user of, in the same way weston is also user of it (libweston-desktop) and it's using it for implementing its desktop-shell. This allows regular xdg-shell clients be managed by the compositor without further interaction and in a transparent manner. Circling back to the Qt-based UI, homescreen will now contain functionality to instruct the compositor to switch between regular XDG toplevel surfaces by using a private protocol called agl-shell. The protocol implementation explicitly assigns roles to different parts of the output: panels (top, left, right, bottom), the background, and desktop (for regular xdg-shell clients). We've adapted homescreen as to use QPA (Qt Platform Abstraction) to bind to the agl-shell protocol interface as to integrate the current functionality we already have in homescreen: a top panel that contains a list of applications and a bottom panel that contains volume UI. Each of these represents a surface and it is a 1-to-1 assignment from a QML Window type, including the background. In terms of desktop role surfaces (or regular XDG clients) the mediaplayer, the launcher, hvac have suffered some slight modification, mostly QML hard-coded width/heigth or the removal of wrappers on top Qt Core GUI. The AGL image is built and deployed in such a manner that the shell is automatically picked from an exported environment variable so the recommendation is that users should just use the regular Qt API when creating new applications and let Qt itself handle it. Do note, that while these application "conversions" should display correctly when using agl-compositor, some of the "regular" functionality might be missing as the focus was towards bring-up of the display part rather than AFM integration. # Further actionable items As some of the functionality disappeared because libhomescreen/libwindowmanager and the homescreen service were no longer necessary, homescreen now acts as UI/window manager/desktop manager and controls, together with the compositor, launching new applications and switching to them. As explained above, the ability to launch application is currently hidden from homescreen by using libhomescreen and the agl-service-homescreen. For the moment, homescreen contains some potential duplicated code (for launching/starting applications) which should probably reside in a generic library which others (like the launcher for instance) can use in a programmatic fashion (and not only for launching applications). Adaptation of the launcher code also has this same duplication part when retrieving the list of runable applications. The activation of top level XDG surfaces is currently done from within homescreen, homescreen being just an example on how that can be achieved. Vendors (and users alike) can modify, just like the compositor, and adapt it as they seem fit. The need to activate/display other applications (but without homescreen involvement) lead us to add an additional protocol extension, which we called agl-shell-desktop. Regular applications can bind to it and receive application_id advertisements whenever new regular XDG application have been mapped in the compositor but also to switch to/activate them. This should allow to preserve the same functionality we currently have in AGL, respectively in agl-service-homescreen. As it stands now there is no code to handle it in the compositor, but will probably make its way to the compositor quite fast once next is back-merged into master. Some of those changes reside in a sandbox repository @ https://gerrit.automotivelinux.org/gerrit/gitweb?p=src/agl-compositor.git;a=shortlog;h=refs/heads/sandbox/mvlad/next-wip To conclude, we split the launching/spawning part of applications, which we believe should be handled independently by using a regular library to talk with afm-system-daemon and the activation part. The activation/switching part, can be handled from within agl-shell, while for regular XDG application should be done using agl-shell-desktop. We could probably make this even clearer by having agl-shell just deal with surface roles and agl-shell-desktop for handling switching of applications but for the time being they will be independent of each other. # How to test/use for the time being ## Building In order to test the compositor and with it, the Qt UI (homescreen), one must enable the 'agl-compositor' DISTRO_FEATURE. This requires a regeneration of the conf/local.conf file so when sourcing the aglsetup script you must pass the -f environment flag: $ source meta-agl/scripts/aglsetup.sh -f -b $MACHINETYPE agl-demo agl-hmi-framework agl-compositor where MACHINETYPE is the machine you're building for. This will build the compositor from master branch and will switch homescreen and apps like mediaplayer/hvac/mediaplayer/launcher to a sandbox repository which contains the converted/adapted applications. Obviously you will need to use bitbake and generate a workable image. If there's no intermediary breakage you should see the regular homescreen starting up upon booting the image. Both homescreen and the regular applications are using sandbox/mvlad/agl-composositor as the integration branch for usage with agl-compositor, and can be pulled individually as well from the following links: - homescreen: https://gerrit.automotivelinux.org/gerrit/gitweb?p=apps/homescreen.git;a=shortlog;h=refs/heads/sandbox/mvlad/agl-compositor - hvac: https://gerrit.automotivelinux.org/gerrit/gitweb?p=apps/hvac.git;a=shortlog;h=refs/heads/sandbox/mvlad/agl-compositor - mediaplayer: https://gerrit.automotivelinux.org/gerrit/gitweb?p=apps/mediaplayer.git;a=shortlog;h=refs/heads/sandbox/mvlad/agl-compositor - launcher https://gerrit.automotivelinux.org/gerrit/gitweb?p=apps/launcher.git;a=shortlog;h=refs/heads/sandbox/mvlad/agl-compositor Note that these require quite a few additional dependencies so the easiest is to to use yocto directly and qemu as MACHINETYPE. ## Start/stop agl-compositor The DISTRO_FEATURE 'agl-compositor' will replace the path of the compositor (currently weston) in the weston unit systemd file to that of agl-compositor, so you should start/stop the compositor using the same unit systemd file (weston). ## Switching application by by default Applications that are not started by homescreen, will not be switched to/activated by default, so we've added an option to have this happening by default, but you need to create a agl-compositor.ini file edit add the the following: [shell] activate-by-default=true Currently, you'll need to add this agl-compositor.ini file argument list in the unit systemd for the weston systemd service file and reload it in order for the compositor to be aware of it. Take a look at the log file to see if the compositor loads the config file accordingly. With this activated, you should be able to start applications, as agl-driver user, using afm-util and they should be activated/switched to upon starting. Note that the compositor is aware of other sections like '[output]' which are required in case you need to change the orientation of the output (just like weston). ## Black surface and delayed presentation As long as there's no client binded to the agl-shell interface or if the client unbinded/has been closed, a black surface will be displayed. This is intentional and is a perfect example for instance if vendors would like to modify this behaviour or not. Also, the agl-shell protocol will delay presentation until the client shell sends a 'ready' request. For more details about the protocol I'd suggest taking a look at: https://gerrit.automotivelinux.org/gerrit/gitweb?p=src/agl-compositor.git;a=blob_plain;f=protocol/agl-shell.xml;hb=refs/heads/master ## Local testing agl-compositor can be built and install locally, with the following being needed: - wayland-protocols - weston (depending on the branch 6 or 7) while the compositor itself can be found at: https://gerrit.automotivelinux.org/gerrit/admin/repos/src/agl-compositor Note that multiple backends exists, like x11 and wayland (nested), but also a native DRM backend, so you should be able to test it directly under different set-ups. For the adventurous, but also for further details regarding client examples: - qtshell (homescreen like simple example) https://gitlab.collabora.com/mvlad/qtshell - to-activate (mimics a regular XDG application) https://gitlab.collabora.com/mvlad/to-activate - activator (mimics a regular XDG application but uses another extension, agl-shell-dekstop to activate other apps like ``to-activate'' above app) https://gitlab.collabora.com/mvlad/activator These client applications do require Qt and some additional AGL packages: libqtappfw and afb-helpers-qt which can be built locally or installed from other parts (depends on your package manager/distribution) but require less dependencies that homescreen or other regular applications. The repositories contain a yocto recipe but they have to be adapted a bit to make them work correctly. # In the future For now the focus was Qt and Qt based UI, but we'll start looking soon into WAM (WebApplicationManager) to determine what would be needed to make it work with agl-compositor and into the policy manager from the windowmanager service and see how that can be adapted for using it with/within the compositor. The Qt application would also need some up-do-date changes as they seem a bit outdated at the moment (this was done around Christmas time) and are in need of further validation of functionality. -- Marius Vlad -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#8112): https://lists.automotivelinux.org/g/agl-dev-community/message/8112 Mute This Topic: https://lists.automotivelinux.org/mt/71400315/2167316 Group Owner: agl-dev-community+owner@xxxxxxxxxxxxxxxxxxxxxxxxx Unsubscribe: https://lists.automotivelinux.org/g/agl-dev-community/leave/4543822/883735764/xyzzy [list-automotive-discussions82@xxxxxxxxxxx] -=-=-=-=-=-=-=-=-=-=-=-
Attachment:
signature.asc
Description: OpenPGP digital signature
