Welcome to the hell at $DAYJOB :). Some comments below..... On Fri, Jun 17, 2022 at 4:56 PM Kevin Fenzi <kevin@xxxxxxxxx> wrote: > > So, when moving openshift apps this week, I really came to the > conclusion we should look at writing up some guidelines for openshift > apps. Perhaps it could have 'MUST' items and 'SHOULD' items and > 'SUGGESTION' items. Yep, sounds good. We have something that we call the "big rules" which fit some of the things here. > > I know I am coming at this from the operations side of things, so please > do share other viewpoints, but let me mention a few issues I see: > > It's very difficult to tell what an application is running on: > - We should probibly forbid 'X:latest' because what the application is > running depends on when it was last built/deployed. (unless: see > below) Yep. It's a bad practice to use "latest", especially for an image that you didn't build. This leaves you with zero control over potentially breaking changes > - A number of apps use images elsewhere (quay.io, etc). We have very > little way to tell whats in those or how they are made or if anyone is > still making them. This is an advantage that we have being a large enterprise - our clusters have no Internet access and therefore all images come from a registry that we control. Perhaps something similar would be valuable here. > > We have a few apps that rebuild/deploy on git changes, which is fine, > but doesn't do anything to keep their container env up to date/secure. One thing that's possible to fix this is to use vulnerability scanning products. We use a proprietary one, but just recently Red Hat open-sourced Stackrox (a company that was acquired a bit ago - https://stackrox.io). I haven't used either the upstream or downstream (Advanced Cluster Security for Kubernetes) version of it (though I did build the upstream version just to see that it worked, it's somewhat difficult to build and AFAIK there's no distribution mechanism/prebuilt images for it - they threw some source code over the wall AFAICT and are looking to improve the build process "soon" so that they can ship prebuilt images - i.e., the build currently requires Docker and doesn't seem to work with podman-docker (I tried.....) > We have a number of apps that tigger on imagestream changes, but as far > as I can tell thats only changes in their image ('bodhi-base') not the > base image that it was built from ('fedora:36'). I'd like to see us > track base images and build/deploy when one changes. If we do that then > :latest might be ok to use, since we know automation took care of > updating it for us. Once you figure that out, let me know :). Bonus points if it doesn't use imagestreams. You still don't want :latest though :). > There's a number of applications using ubi (rhel's base image), which I > think might be worth recommending, especially if we start build/deploy > on base image changes. Likely Fedora base images will change more > quickly. Yep, this is one of our "big rules" that we prefer ubi-based images. For internally built code, we require it and for vendor apps we prefer it. Since I imagine that we don't have many "vendor" apps (except maybe stuff like Redis, which already exists as a Red Hat packaged downstream image which should likely be preferred), the amount of non-ubi images should be minimal I would think. > Currently we have some apps that were setup and probibly have not > changes in years, happily running the same old container image since > there was no reason to redeploy them. Yep :( One of our requirements is that a pod lifetime is no longer than 2 weeks (we'll delete any pod older than 2 weeks, and the underlying controller will recreate it). We also won't allow images with a certain vulnerability rating to run, so you're pretty assured that you're running current stuff. This might be a bit draconian for infra, but something to consider/think about. > Similarly, python versions are all over the place... python3.6, 3.8, > 3.9, etc. Of course different apps will target different versions > sometimes, but it might be nice to make sure we are using a supported > version by whatever container image the app is using. (ie, gets security > fixes). See above, once you have unpatched vulnerabilities you can't run, so this is effectively mitigated. > Anyhow, I'll stop there... but hopefully everyone would be ok with such > a doc and then changing things to match it? Doc makes sense, and I might steal some of it :). _______________________________________________ infrastructure mailing list -- infrastructure@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to infrastructure-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
