On Wed, Jul 4, 2018 at 8:46 AM, Fabiano Fidêncio <fabiano@xxxxxxxxxxxx> wrote: > On Tue, Jul 3, 2018 at 1:54 PM, Daniel P. Berrangé <berrange@xxxxxxxxxx> wrote: >> On Tue, Jul 03, 2018 at 01:32:28PM +0200, Fabiano Fidêncio wrote: >>> Folks, >>> >>> One thing that has been discussed between Felipe Borges (GNOME Boxes >>> maintainer) and myself is the possibility to, from an app, download >>> and install the latest osinfo-db content without being dependent of >>> distribution packagers and being able to ensure that the users will >>> have the most up-to-date db as soon as a something was >>> introduced/changed. >>> >>> In order to do so, we'd have to (according to my understanding): >>> - Support URLs as arguments for osinfo-db-import: >>> This step would allow the apps to download the latest release from >>> our official source; >> >> This is fine, and needed no matter what else is done I think. >> >>> - Expose a "latest" osinfo-db build: >>> Currently we tag our releases, make a tarball and this tarball is >>> uploaded in pagure. The tarball's name looks like >>> "osinfo-db-20180612.tar.xz" and, preferably, we'd like to have >>> something like "osinfo-db-latest.tar.xz" pointing to the latest >>> release. Actually, in an ideal world, we'd like to have a >>> osinfo-db-latest.tar.xz pointing to the last osinfo-db commit (then, >>> maybe, we would have to rely on gitlab infra to do that?); >> >> I think the idea of a -latest.tar.xz link is a mistake as it >> encourages apps to download the content even if hasn't been changed, >> as they can't easily see if it is a new version or not. > > With the -latest naming I was envisioning that we could ship > non-official releases, but git master. Argh. "(...) non-official releases, like git master." Sorry for the typo. > So, first thing, the name is, at least, misleading. :-/ > > Daniel, > You think that shipping git master would also be a mistake? We can > easily do a release per day we have patches and update the release in > pagure without issues. But it becomes problematic in order to how > distribute those to fedora (considering that the time waiting for > those to reach the distro is ~2 weeks from the release day ... and I'm > packaging it as soon as a release is done). > > If you think this idea is okay, how do you envision it being > shipped/displayed to the apps to download? The reason I'm asking here > is because if we take this path, we'd have to have a way to provide > git master already built to the apps and having this on pagure, for > sure, doesn't seem like a good idea. > >> >>> - Provide a DBus API for osinfo-db-import and osinfo-db-validate (at least): >>> With this we can easily spawn, for instance, `osinfo-db-import >>> https://releases.pagure.org/libosinfo/osinfo-db-latest.tar.xz` from >>> any app using osinfo-db; >> >> I'm pretty sceptical about need for adding dbus into this. I don't see >> why an app couldn't just spawn the commands as is. Turning them into >> a DBus service feels like massive overkill for the problem. > > Okay from my side. > I'd like to hear here from Felipe as well why a dbus API would be > their preferred way and, mainly, whether we could go for spawning the > command as they are now. > >> >> Also we must not have any applications hardcoding releases.pagure.org >> URLs. I do not have confidence in pagure.org sticking around over the >> long time frames RHEL lives for. >> >>> Ideas? Possible limitations? >> >> I've been thinking about live updates for quite a while - ever since I >> created the osinfo-db-tools separate package in fact :-) >> >> The two key problems with any live update mechanism are scalability and >> compatibility. We need to ensure that applications don't repeatedly >> hit the web server to download unchanged content over & over. We also >> neeed to ensure that we don't bake in usage of a URL that is liable >> to change over time. Whatever we do will end up in RHEL releases that >> can live for 10+ years, and I don't have confidence our hosting will >> live for 10+ years without moving at least once. >> >> My thought was to leverage DNS as a key part of the solution. Have DNS >> TXT record exposing the current live update URL ie a TXT query against >> "dbupdate.libosinfo.org" could return something akin to >> >> url=https://releases.pagure.org/libosinfo/osinfo-db-20180612.tar.xz,version=20180612 >> >> The locally install osinfo-db content always creates a file VERSION in >> the root of the install location. So that content can be compared against >> the DNS result to see if it is outdated of not, and thus avoid hitting >> the web server at all unless there's genuinely a change that needs >> acquiring. >> >> I would anticipate some osinfo-db-update-check tool to perform this work >> and print out a URL, that could then be given to osinfo-db-import. > > I like the idea! > > Who's the owner of libosinfo.org? Daniel? Red Hat? > > Daniel, > Did you start poking at this in the past? If not, it's something I can > slowly starting to take a look at. > >> >> It feels like a cron job to kick off periodic updates would be nice, but >> the downside with that is that it synchronizes a DOS attack from all >> deployed hosts (modulo timezone differences). > > The idea about what will be used on Boxes is something to be discussed > during its hackfest that will happen next Monday. > I'm expecting that for generating the flatpack a cron job will be > used. I'm not sure whether we want to also have an option to update > the osinfo-db on non-flatpak'ed apps (aka, the ones normally shipped > on OSes), although I do believe GNOME Boxes should! > >> >> Security is, however, a key issue here. For this to be practical at the >> very least it would likely require either DNSSEC, or embedding a crypto >> signature in the result. The latter would however require a long term >> pub key to be embedded in the client, and I'm not a fan of that. >> >> There is a concern of reinventing the wheel here. There is some kind of >> industry standard I vaguely recall for doing software/content updates >> over the internet. I've not investigated that in enough detail to >> understand if it avoids the scalability & compatibility concerns I >> have though > > Hmm. Right, it makes sense. I'll check what the standard is and how we > could take advantage of that. > . >> >> Regards, >> Daniel >> -- >> |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| >> |: https://libvirt.org -o- https://fstop138.berrange.com :| >> |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| > > > Best Regards, > -- > Fabiano Fidêncio -- Fabiano Fidêncio _______________________________________________ Libosinfo mailing list Libosinfo@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libosinfo
