On Mon, Sep 25, 2023 at 1:45 PM Alexander Ploumistos <alex.ploumistos@xxxxxxxxx> wrote: > > On Mon, Sep 25, 2023 at 7:33 PM Artur Frenszek-Iwicki > <suve@xxxxxxxxxxxxxxxxx> wrote: > > > > Without docs, the whole process is a black box. > > +1 > > That was actually one of the two reasons I started this thread. > > For instance, is this the package that must be rebuilt in order to see > an AppStream metadata file appear/refreshed in GNOME Software? > https://src.fedoraproject.org/rpms/appstream-data > Could we get an explainer (for dummies) of how the entire pipeline works? > > The other reason was someone from Debian telling one of my upstreams > "Don't use appstream-util to validate the file, it's deprecated, use > appstreamcli". Frankly, that struck a chord, considering x years ago I > had spent a considerable amount of time convincing dozens of projects > to adopt "AppData" (at the time), filing tickets and submitting PRs. So here's some backstory... Way back in the days of yore (about 10 years ago), people were working on a cross-distro Software Center: https://fedoraproject.org/wiki/Features/SoftwareCenter For that "software center", the AppData (renamed to AppStream) format was created initially to port over desktop files to XML for more reliable parsing at scale. The goal here was to create a cheaper and more reliable method of providing rich metadata in repositories that could be indexed and used to present useful information about applications. My understanding is that the original AppStream implementation at the time was moribund due to its maintainer being in school, so a new one was made (unhelpfully named appstream-glib... why "unhelpfully" you ask? well, the original appstream was *also* glib based!). This got plugged into what became GNOME Software. And because Fedora needed a tool to generate the indexes by scanning all the RPMs, appstream-builder was created to do that. (As an aside, appstream-glib was never used by KDE because there was no Qt/C++ binding for it, but the original AppStream did have one, so that's what Plasma Discover has always used.) But there was a problem: unlike RPM repodata generation, AppStream repodata generation is slow. Really slow. This is because AppStream repodata requires opening up the RPM, plucking data out of it, and applying the necessary transformations per repodata generation policy (downloading remote images, recompression, etc.). There are arguments about whether it's appstream-builder specific (given that Debian's archive is bigger, their package format is more complex to "pluck" from, etc and appstream-generator does fine there), but the point is that for Fedora with appstream-builder, it's too slow. Note that the context of this is back in the era where Fedora composes took half a day or longer. Adding even more hours to that process was not appealing (it would have to be done for every repo being composed for every architecture...). So a stopgap solution was implemented: appstream-data. Richard Hughes maintains a local mirror of the full Fedora repositories and generates the appstream data from that using scripts[1] and extra data[2] to produce the appstream-data package[3]. The goal is to someday get to a point where we can have this done properly as repodata through createrepo_c[4], but we're not able to do that yet. Now, appstream-util is the appstream-glib equivalent of appstreamcli. We test using appstream-util because we need the AppStream data to be valid for appstream-builder to consume it. If it is not, the application will be skipped in the next round of appstream repodata updates. If we ever move to a process that uses libappstream-compose as the backend library for producing AppStream repodata, then we can stop caring about appstream-util. But for now, here we are. [1]: https://github.com/hughsie/appstream-scripts [2]: https://github.com/hughsie/fedora-appstream [3]: https://src.fedoraproject.org/rpms/appstream-data [4]: https://github.com/rpm-software-management/createrepo_c/issues/75 -- 真実はいつも一つ！/ Always, there's only one truth! _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
