On Tue, Jul 03, 2018 at 01:32:28PM +0200, Fabiano Fidêncio wrote: > Folks, > > One thing that has been discussed between Felipe Borges (GNOME Boxes > maintainer) and myself is the possibility to, from an app, download > and install the latest osinfo-db content without being dependent of > distribution packagers and being able to ensure that the users will > have the most up-to-date db as soon as a something was > introduced/changed. > > In order to do so, we'd have to (according to my understanding): > - Support URLs as arguments for osinfo-db-import: > This step would allow the apps to download the latest release from > our official source; This is fine, and needed no matter what else is done I think. > - Expose a "latest" osinfo-db build: > Currently we tag our releases, make a tarball and this tarball is > uploaded in pagure. The tarball's name looks like > "osinfo-db-20180612.tar.xz" and, preferably, we'd like to have > something like "osinfo-db-latest.tar.xz" pointing to the latest > release. Actually, in an ideal world, we'd like to have a > osinfo-db-latest.tar.xz pointing to the last osinfo-db commit (then, > maybe, we would have to rely on gitlab infra to do that?); I think the idea of a -latest.tar.xz link is a mistake as it encourages apps to download the content even if hasn't been changed, as they can't easily see if it is a new version or not. > - Provide a DBus API for osinfo-db-import and osinfo-db-validate (at least): > With this we can easily spawn, for instance, `osinfo-db-import > https://releases.pagure.org/libosinfo/osinfo-db-latest.tar.xz` from > any app using osinfo-db; I'm pretty sceptical about need for adding dbus into this. I don't see why an app couldn't just spawn the commands as is. Turning them into a DBus service feels like massive overkill for the problem. Also we must not have any applications hardcoding releases.pagure.org URLs. I do not have confidence in pagure.org sticking around over the long time frames RHEL lives for. > Ideas? Possible limitations? I've been thinking about live updates for quite a while - ever since I created the osinfo-db-tools separate package in fact :-) The two key problems with any live update mechanism are scalability and compatibility. We need to ensure that applications don't repeatedly hit the web server to download unchanged content over & over. We also neeed to ensure that we don't bake in usage of a URL that is liable to change over time. Whatever we do will end up in RHEL releases that can live for 10+ years, and I don't have confidence our hosting will live for 10+ years without moving at least once. My thought was to leverage DNS as a key part of the solution. Have DNS TXT record exposing the current live update URL ie a TXT query against "dbupdate.libosinfo.org" could return something akin to url=https://releases.pagure.org/libosinfo/osinfo-db-20180612.tar.xz,version=20180612 The locally install osinfo-db content always creates a file VERSION in the root of the install location. So that content can be compared against the DNS result to see if it is outdated of not, and thus avoid hitting the web server at all unless there's genuinely a change that needs acquiring. I would anticipate some osinfo-db-update-check tool to perform this work and print out a URL, that could then be given to osinfo-db-import. It feels like a cron job to kick off periodic updates would be nice, but the downside with that is that it synchronizes a DOS attack from all deployed hosts (modulo timezone differences). Security is, however, a key issue here. For this to be practical at the very least it would likely require either DNSSEC, or embedding a crypto signature in the result. The latter would however require a long term pub key to be embedded in the client, and I'm not a fan of that. There is a concern of reinventing the wheel here. There is some kind of industry standard I vaguely recall for doing software/content updates over the internet. I've not investigated that in enough detail to understand if it avoids the scalability & compatibility concerns I have though. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| _______________________________________________ Libosinfo mailing list Libosinfo@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libosinfo
