On Wednesday, May 17, 2023 6:00:01 PM EDT Dan Mick wrote: > A long long time ago when this first came up, I suggested we have > instructions that mechanized finding the package from download.ceph.com > and downloading/extracting the zipapp from it. I felt strongly at the > time that we should try our best to avoid having two sources of truth > for the Real Tested Binary, and packages were the obvious answer, but > using the actual package management software was not suitable, so wrote > the extraction code. > > We didn't ever release that code for extracting from the package, but > the technique found its way into the build jobs in Oct 2021: > > At present, at least for the CI builds, the cephadm binary/zipapp is > extracted from packages during the build and placed, separately, in the > package repo as a plain file, unmanaged by the package managers. See > https://github.com/ceph/ceph-build/blob/main/ceph-dev-new-build/build/build_ > rpm#L59-L63 and > https://github.com/ceph/ceph-build/blob/main/scripts/build_utils.sh#L984-L98 > 8 (the asymmetry was there when I got there). OK, that's a good start. :-) > > That code is also present in the "ceph-dev-build" and "ceph-build" jobs > (which do candidate builds for named releases and final release builds, > respectively), and indeed cephadm is present in the rpm repos (for > example, http://download.ceph.com/rpm-17.2.6/el8/noarch/cephadm) but I > don't see the cephadm binary on download.ceph.com for deb repos. (It's > probably the "sync to download.ceph.com" step being unaware it should > copy anything besides .deb.) Unfortunately, I don't think this RPM derived cephadm is appropriate for general uses. RPM mangles the shebang line to something RH-distro specific so this `cephadm` wouldn't work properly on distros like debian, ubuntu, etc. I think it does this to the zipapp too - but I would need to verify it. I'm aware of chacra.ceph.com (and 1.chacra.ceph.com) but all I can get from it is JSON and I would like to inspect the actual binary. I'm probably not using the proper interface but I don't know where to look. If the file extracted from the ubuntu deb package does not get mangled and the shebang simply refers to `python3` then we'd be good there once the issue with the deb release process is resolved. > > My suggestion would be to debug the issue with the deb release process, > and document that the binary can be found in the appropriate path > corresponding to the release in download.ceph.com. Sounds like a plan, is that something you expect us to do, or is it something you are planning to do? I'm happy to help but I'm afraid I wouldn't know where to start so at least I'd need some initial "hand holding". Thanks! > > On 5/16/2023 10:10 AM, John Mulligan wrote: > > Hi Dan & others, > > > > A few months back we (the Orchestration team working on cephadm), > > discussed > > the new compiled [1] cephadm in a CLT call and we briefly discussed it on > > the list [2]. I wanted to revisit that conversation as Reef's release is > > quickly coming up. > > > > To summarize: > > Previous versions of Ceph contained a single python source file for > > cephadm. Upstream users were instructed to copy this source file directly > > out of the Ceph git tree and execute the file using python. Now, the ceph > > build process creates an executable python zipapp from the source file > > and the old instructions are out of date. > > > > We would like to request assistance with making it possible for users who > > are planning on bootstrapping clusters to download an already compiled > > version of cephadm from a canonical location. A secondary goal would also > > be to sign that binary. However, I would treat that as a nice to have > > since the current workflow doesn't have this. > > > > I'd love to discuss the technical aspects of this and get some code to > > implement this in place. I'd be happy to continue the conversation here - > > or if you prefer - in the discussion forum of your choice. Thanks very > > much! > > > > [1] - It is still python, and it's not compiled to native binaries but is > > rather a zipapp - https://docs.python.org/3/library/zipapp.html > > > > [2] - I wanted to link to the previous discussion but neither the > > Archived-At header url or search turn it up. So maybe it got lost in the > > infra issues a while back. If you need references to the previous thread, > > I can forward them because I still have them on my mailserver. > > > > > > _______________________________________________ > > Dev mailing list -- dev@xxxxxxx > > To unsubscribe send an email to dev-leave@xxxxxxx > > _______________________________________________ > Dev mailing list -- dev@xxxxxxx > To unsubscribe send an email to dev-leave@xxxxxxx _______________________________________________ Dev mailing list -- dev@xxxxxxx To unsubscribe send an email to dev-leave@xxxxxxx
