Adam Williamson <adamwill@xxxxxxxxxxxxxxxxx> writes: > On Thu, 2022-07-07 at 10:49 -0700, Stewart Smith via devel wrote: >> We actually have a skeleton design for such a thing (it says what >> updates and upgrades are available), but we've lagged on >> both posting to devel@ that it's something we've been working on, and I >> need to go and poke the person who needs to click the "publish this repo >> to github" button for the DNF plugin. I'll go do that now, as it would >> certainly add to this conversation. > > The problem with this is that it's a very fuzzy, messy space that's > hard to put limits and requirements on. > > The URL you mentioned is a case in point. That has more or less assumed > the status of running joke among a small group of people who care about > it. When you go to that URL, what you're getting now is a static JSON > document that is hand-maintained in the fedora ansible tree: > > https://pagure.io/fedora-infra/ansible/blob/main/f/roles/pkgdb-proxy/files/pkgdb-gnome-software-collections.json > > when a release is branched, released, or EOLed, somebody from releng > has to remember to edit that file, and get the edit right. These days > that...usually...happens. I cannot possibly imagine a process which relies on good intentions ever failing :) > Way back in the mists of time, what you got at that URL was a view on > the release database of the pkgdb app, but that app hasn't existed for, > uh, quite a lot of years now. When it was decommissioned, we discovered > several things were relying on that API endpoint, which we replaced > with a static JSON file just until it could be replaced with something > less dumb. Yup, still here. > > The original planned replacement, IIRC, was PDC, which has now had an > entire lifecycle (conceived, created, deployed, given up on, and nearly > retired) without ever fully implementing all the release lifecycle > tracking bits that various pkgdb consumers wanted. (It's been a while, > but IIRC, there were issues with distinguishing between 'EOL', 'current > stable', 'branched' and 'rawhide' releases from PDC data). > > There are various sources of some kind of truth regarding the Fedora > release cycle. Bodhi has one - you can query > https://bodhi.fedoraproject.org/releases/ with content-type JSON and > you'll get some JSON back with some properties of various things Bodhi > considers to be "releases", including a "state". I've been converging on the thought that the same source of truth needs to be the input to the big things on the web site and docs, and well integrated into a regular release process that's as automated as humanly possible. By release, I mean "released any kind of package update" - i.e. the thing you do *all the time*. There's still the problem of people remembering to update it, but with fewer places to change it, at least it's easier to get them all. > I maintain one which is used by my 'fedfind' tool/library that various > bits of infra (mostly QA things, but also a few others) use. It is a > *different* hand-edited bit of JSON, which I created after being > annoyed at the collections one being updated wrongly a few times: > https://fedorapeople.org/groups/qa/metadata/release.json > > There's the CurrentFedoraVersion template on the wiki: > https://fedoraproject.org/wiki/Template:CurrentFedoraVersion > > There's also, I think, some variables for 'current' releases in the > Fedora infra ansible deployment. > > There are some fun issues you run up against when you play around in > this sandbox long enough. One, for instance - when *exactly* is a new > Fedora release "released"? Is it when the 'fedora' repo is frozen and > Bodhi is kicked over to pushing new 'stable' updates to the 'updates' > repo instead? (That's Bodhi's definition). Is it when the releases/NN > tree on the public mirror system is available? (That's the one I use, > more or less, for fedfind). Is it when the release announcement goes > out? The point here being, if you think about it hard enough, the > question of what 'states' a distro (or any software product) can be in > is quite a difficult one and probably has a lot of answers. We have a similar challenge even just entirely within AWS. Our mirror model is very different from a typical linux distribution. We're a completely push model rather than mirrors being asynchronous copies. We do it this way so that we can be *certain* when updates hit different locations, as well as control the release velocity, and take action if we discover new sadness during a release. It's the balance between: 1. Customers are frustrated if they know about a package that fixes a security bug but it isn't available in the region where they run their service. 2. Not wanting to have a blast radius of "all of EC2" when there's a problem with an update. Due to our push based mirror model, we can follow the ethos of "once you know about a release, you can get it anywhere you run Amazon Linux" and have a different distribution rate for the different places where we need the content of the OS itself to exist, and the release notification metadata. For Amazon Linux 1 and 2 it's still a bit fuzzy as the default on launch behavior is to apply Critical and Important security updates, so as soon as they're in a local repo everything newly launched gets them. So there's still an "eventually consistent" model with those OSs between the web page listing security updates (ALAS), the release notes (documentation), and what package updates are available to instances. This is a bit easier in an Amazon Linux 2022 world where we lock instances to a specific snapshot (version) of the repositories and have a mechanism to move which version an instance points to. With the AL2022 world we can properly order: 1. distribute the new version of the repositories everywhere 2. release release notes and security updates web site 3. update the release metadata file to point to the new version. 4. release new AMIs and container images, updating all the pointers to them Thus, anyone hitting refresh on the web sites in (2) can get the new packages as the content is already everywhere. And 3 and 4 are also just changing the advertising of the updates that are already available to be pulled if you know what version to ask for. Without that push model for mirrors though, it gets a bit more fuzzy - especially if that latency can be a decent amount of wall clock time. One advantage of this metadata being part of the repository is that the data travels with the repository as you roll out updates, and if you know roughly when the EoL will be, you at least won't be too wrong if the mirror is really stale. > Another - what do we do about, e.g., Fedora IoT and Fedora CoreOS, > which have their own somewhat different release/life cycles? What about > module lifecycles? What is it about *lifecycles* that's important, > anyway? Don't we maybe want to just have a sort of generic system for > "important events"? I view it as a mechanism to communicate well in advance of when someone is going to have to do work. Fedora is the simple case: every 6-12 months you're going to have to upgrade the version of the OS. For things like Modules, having that communication be clear can be a benefit too. Some knowledge of what life cycles can mean could be useful for setting policy. You could better reason about how loudly to alarm on an encrouging EoL. A new version of the OS being available could be more Informational, while your current version moving to "critical updates only" may be a lot louder, and "full EoL next week" could be "start paging people". > I kinda have a dream here, where there would be one canonical system > for registering 'Fedora-related events', and all 'truths' about what > releases are stable and EOL and whatknot in various systems, along with > lots of other stuff, could be derived from it. That thing would be a > really dumb list of 'events', where an event is really just something > like this: > > "thing X reached state Y" > > where X and Y are both free text. The system would publish a message > for each event, and it could also be queried. So when a Fedora release > went EOL, as part of the SOP for doing that, an event: > > "fedora36 reached state eol" > > or something like that would be created, the system would publish that, > and anyone who wanted to figure out what releases are eol could query > the system for all things in state 'eol' and work it out. You could > *also* use the system for things like: > > "fedora36 reached state stringfreeze" > > or whatever. I also want to hook it into every security scanner under the sun so that upcoming EoL can be just as alarming as any random vulnerability, and thus allowing people to set policy such as "treat running an EoL OS the same as running something that is unpatched with a Critical vulnerability", and warn people well in advance. Mind you, a stream of notifications that can be consumed is probably also needed, as then you can trigger various automated things off it, e.g. "try a build+test run on the new release". Thus, it could work really well if the way you triggered the change of state was to update the input into such a notification system rather than changing anything else anywhere. i.e. the way you make "fedora 36 reached state stringfreeze" happen would be to get the pull request merged to this location, which then fires off all the things to both do that and tell people about it. > I built a very dumb implementation of this: > https://pagure.io/fedora-qa/releasestream > > and meant to try and improve it a bit and get it deployed and then > start trying to get people to actually use it for stuff, but ran into > ENOTIME issues. I'm still hoping to get back to it at some point... > -- > Adam Williamson > Fedora QA > IRC: adamw | Twitter: adamw_ha > https://www.happyassassin.net > > _______________________________________________ > devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx > To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx > Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx > Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
