On Thursday, July 18, 2024 2:53:37 PM EDT Casey Bodley wrote: > reviving an old thread, but i'm still very interested in this idea. > has any other progress been made here? are there simple steps we could > take to start experimenting with this? Hi there, Everything I have with regards to this is available to look at via https://github.com/ceph/ceph/pull/56711 I periodically rebase it and occasionally use it to build ceph. However, lately I have been too busy working on the smb support to spend much time looking at this. Happy to continue discussing this effort and sharing my experiences and thoughts but I won't have a lot of time to spend on it directly for a while - probably a month or so. > > On Thu, Aug 18, 2022 at 6:19 AM Ernesto Puerta <epuertat@xxxxxxxxxx> wrote: > > > > > > Hi John, > > > > > > > >> I'm not very jenkins savvy so I can't speak much to that part. I'll dig > >> into your dockerfile a bit. One thing I note is that there's only a > >> dockerfile for centos (stream). The build infrastructure I'm imagining > >> let's one build ubuntu binaries & packages and/or centos binaries & > >> packages regardless of the "real" OS/distro . What do you think about > >> supporting multiple different dockefiles for each supported distro and > >> selecting between them hypothetically based on a (distro, release, > >> flavor, arch) style tuple like used in the builds?> > > > > > > > > Yeah, that was my idea too. We currently support 2 different distros in > > our CI (CentOS and Ubuntu), but we randomly test a PR only on one of > > them. With containerized (reproducible) builds we could start using a > > distributed ccache, and with the savings in build times we could afford > > running the PR pipeline for 2 different distros. > > > > > > > Regarding the Dockerfile, if you check it, there's only 1 line of code > > that is CentOS-specific (the "dnf install" for the EPEL repo). If we move > > that to the install-deps.sh, we could have a distro-neutral Dockerfile > > (the base layer would still have to be set via DISTRO build arg). > > > > > > >> > >> > >> > Kind Regards, > >> > Ernesto > >> > > >> > > >> > > >> > > >> > On Fri, Aug 12, 2022 at 5:20 PM John Mulligan > >> > <phlogistonjohn@xxxxxxxxxxxxx> wrote: > >> > > >> > > On Thursday, August 11, 2022 7:17:19 PM EDT John Mulligan wrote: > >> > > > >> > > > On Thursday, August 11, 2022 5:34:40 PM EDT Josh Durgin wrote: > >> > > > > >> > > > > I think it's a great idea - it's related to ideas in this > >> > > > > thread: > >> > > > >> > > https://lists.ceph.io/hyperkitty/list/dev@xxxxxxx/thread/UD43OL6YBN5A > >> > > 2QHLK > >> > > > >> > > > >> > > > >> > > > > RU QLYQRXMHM5FKJ/ > >> > > > > >> > > > > >> > > > > >> > > > Indeed, I remember participating a little in that thread. > >> > > > > >> > > > > >> > > > > >> > > > > The main idea there is to make it simple to update containers so > >> > > > > you > >> > > > >> > > > >> > > > >> > > can > >> > > > >> > > > >> > > > >> > > > > run teuthology tests against code you just changed with very > >> > > > > little > >> > > > > overhead (no need to wait for hours for package + container > >> > > > > complete > >> > > > > rebuilds). > >> > > > > >> > > > > >> > > > > >> > > > Great, I've been thinking about this as well - I've been poking at > >> > > > cpatch > >> > > > here-and-there as well and had started working on a python version > >> > > > that > >> > > > I > >> > > > believe will be easier to hack on further. I need to polish it up > >> > > > a > >> > > > >> > > > >> > > > >> > > little > >> > > > >> > > > >> > > > >> > > > and share it. > >> > > > >> > > > >> > > > >> > > I spent some time today to make my branch presentable and filed an > >> > > RFC PR > >> > > for > >> > > the python version of cpatch I mentioned above. Posting it here for > >> > > context: > >> > > https://github.com/ceph/ceph/pull/47573 > >> > > > >> > > > >> > > > >> > > > > Zack Cerza has made a lot of progress on the teuthology side of > >> > > > > this - > >> > > > > running the tests locally using an existing container image. The > >> > > > > 2nd > >> > > > >> > > > >> > > > >> > > half, > >> > > > >> > > > >> > > > >> > > > > of making it easy and fast to update a container image, is still > >> > > > > TBD. > >> > > > > >> > > > > >> > > > > >> > > > That's great! I only got Teuthology in Sepia access recently and > >> > > > I > >> > > > would > >> > > > love to try out this version too. Is there a link to a WIP PR or > >> > > > >> > > > >> > > > >> > > something > >> > > > >> > > > >> > > > >> > > > along that line? I'd be interested in trying it out a bit. > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > In the short term, I'll try to put together a proof-of-concept PR > >> > > > for > >> > > > >> > > > >> > > > >> > > some > >> > > > >> > > > >> > > > >> > > > of the build container ideas I'm thinking of. It seems like > >> > > > there's a > >> > > > >> > > > >> > > > >> > > fair > >> > > > >> > > > >> > > > >> > > > amount of interest. > >> > > > > >> > > > > >> > > > > >> > > > Thanks! > >> > > > > >> > > > > >> > > > > >> > > > > Josh > >> > > > > > >> > > > > > >> > > > > > >> > > > > On Thu, Aug 11, 2022 at 12:09 PM Tom R < > >> > > > >> > > > >> > > > >> > > precision.automobilia@xxxxxxxxx> > >> > > > >> > > > >> > > > >> > > > > wrote: > >> > > > > > >> > > > > > John > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > I think your proposal to separate the build process such that > >> > > > > > the > >> > > > >> > > > >> > > > >> > > user > >> > > > >> > > > >> > > > >> > > > > > can > >> > > > > > select an os flavor of their liking is a fantastic idea. > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > I'm not familiar with the process to assist but would love to > >> > > > > > follow > >> > > > >> > > > >> > > > >> > > if > >> > > > >> > > > >> > > > >> > > > > > this proposal is accepted. > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > On Thu, Aug 11, 2022, 1:37 PM John Mulligan > >> > > > > > <phlogistonjohn@xxxxxxxxxxxxx> > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > wrote: > >> > > > > > > >> > > > > >> On the user's list one thread about the packages took a turn > >> > > > > >> into > >> > > > > >> discussing > >> > > > > >> building in containers [1]. This is a topic that I have had > >> > > > > >> some > >> > > > >> > > > >> > > > >> > > idle > >> > > > >> > > > >> > > > >> > > > > >> conversations about with Adam King and so I figured that I > >> > > > > >> would > >> > > > >> > > > >> > > > >> > > raise > >> > > > >> > > > >> > > > >> > > > > >> this to > >> > > > > >> a wider audience. > >> > > > > >> > >> > > > > >> > >> > > > > >> > >> > > > > >> My thought is to use container images specifically for > >> > > > > >> building > >> > > > >> > > > >> > > > >> > > ceph - > >> > > > >> > > > >> > > > >> > > > > >> and not > >> > > > > >> just it's container images. The builds may continue to > >> > > > > >> produce > >> > > > > >> packages, > >> > > > > >> but a > >> > > > > >> container would be used as an abstraction between the actual > >> > > > > >> OS and > >> > > > >> > > > >> > > > >> > > the > >> > > > >> > > > >> > > > >> > > > > >> build > >> > > > > >> process of (do_cmake.sh, etc.). > >> > > > > >> > >> > > > > >> > >> > > > > >> > >> > > > > >> Builder images would be be available for use both by the > >> > > > > >> build > >> > > > >> > > > >> > > > >> > > system > >> > > > >> > > > >> > > > >> > > > > >> (jenkins > >> > > > > >> builders) as well as individual users. One advantage of this > >> > > > > >> is > >> > > > > >> that > >> > > > > >> the > >> > > > > >> user > >> > > > > >> can build the packages for distros that don't match the local > >> > > > >> > > > >> > > > >> > > distro. > >> > > > >> > > > >> > > > >> > > > > >> I've > >> > > > > >> also found it advantageous for my own builds to use the > >> > > > > >> container > >> > > > > >> to > >> > > > > >> limit > >> > > > > >> memory and CPU for the build. > >> > > > > >> > >> > > > > >> > >> > > > > >> > >> > > > > >> I'm curious if anyone has discussed this before. Does it > >> > > > > >> interest > >> > > > > >> anyone? I > >> > > > > >> am willing to volunteer some time to help as well. > >> > > > > >> > >> > > > > >> > >> > > > > >> > >> > > > > >> Thanks for reading! > >> > > > > >> > >> > > > > >> > >> > > > > >> > >> > > > > >> > >> > > > > >> [1] > >> > > > >> > > > >> > > > >> > > https://lists.ceph.io/hyperkitty/list/ceph-users@xxxxxxx/message/ > >> > > > >> > > > >> > > > >> > > > > >> VR3ZKP4T2PLZ6BJ23GPZAG3KBV6AI3LA/ > >> > > > > >> < > >> > > > >> > > > >> > > > >> > > https://lists.ceph.io/hyperkitty/list/ceph-users@xxxxxxx/message/VR3Z > >> > > K > >> > > > >> > > > >> > > > >> > > > > >> P4 > >> > > > > >> T2PLZ6BJ23GPZAG3KBV6AI3LA/> > >> > > > > >> > >> > > > > >> > >> > > > > >> > >> > > > > >> > >> > > > > >> _______________________________________________ > >> > > > > >> Dev mailing list -- dev@xxxxxxx > >> > > > > >> To unsubscribe send an email to dev-leave@xxxxxxx > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > _______________________________________________ > >> > > > > > Dev mailing list -- dev@xxxxxxx > >> > > > > > To unsubscribe send an email to dev-leave@xxxxxxx > >> > > > > >> > > > > >> > > > > >> > > > _______________________________________________ > >> > > > Dev mailing list -- dev@xxxxxxx > >> > > > To unsubscribe send an email to dev-leave@xxxxxxx > >> > > > >> > > > >> > > > >> > > _______________________________________________ > >> > > Dev mailing list -- dev@xxxxxxx > >> > > To unsubscribe send an email to dev-leave@xxxxxxx > >> > >> > >> > >> > >> > > _______________________________________________ > > Dev mailing list -- dev@xxxxxxx > > To unsubscribe send an email to dev-leave@xxxxxxx > > _______________________________________________ > Dev mailing list -- dev@xxxxxxx > To unsubscribe send an email to dev-leave@xxxxxxx _______________________________________________ Dev mailing list -- dev@xxxxxxx To unsubscribe send an email to dev-leave@xxxxxxx
