Hi Tom! Thanx a lot of for input. Although I did my bes with the tagging, it will be learning on the go. I had adapted https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#theoretical_tagging_solution as you suggested. It is great improvement. Especially the config, I was not aware about. That woudl indeed help a lot. The usage of pernament tag is someging I have to learn, but is moreover necessary for proper srpm rebuil. TYVM! J. On Wed, 28 Jun 2023 at 21:31, Tom Stellard <tstellar@xxxxxxxxxx> wrote: > > On 6/26/23 09:21, Aoife Moloney wrote: > > https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#including_portable_srpms_in_release_(improving_of_step_6) > > > > This document represents a proposed Change. As part of the Changes > > process, proposals are publicly announced in order to receive > > community feedback. This proposal will only be implemented if approved > > by the Fedora Engineering Steering Committee. > > > > > > == Summary == > > > > This is the last step in > > https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs > > effort. JDKs in fedora are already static, and we repack portable > > tarballs into RPMs. Currently, the portable tarball is built for each > > Fedora and EPEL version. Goal here is to build each JDK > > (8,11,17,21,latest (20)) only once, in oldest live Fedora repack in > > all live Fedoras. If jdk is buitl in epel, it will be built in oldest > > possible epel and repacked in newer live epels. > > > > > > == Owner == > > * Name: [[User:jvanek| Jiri Vanek]] > > > > * Email: jvanek@xxxxxxxxxx > > > > > > == Detailed Description == > > > > As described in > > https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs ; > > during last year, packaging of JDKs had changed dramatically. As > > described in the same wiki page and in individual sub changes and > > devel threads, the primary reason for this is to lower maintenance and > > still keep Fedora Java friendly. > > > > * In the first system wide change, we have changed the JDKs to build > > properly as standalone, portable JDK - the way JDK is supposed to be > > built. I repeat, we spent ten years by patching JDK to become properly > > dynamic against system libs, and all patches went upstream, but this > > has become a fight which can not be won. > > > > * As a second step we introduced portable RPMs, which do not have any > > system integration, only build JDK and pack the final tarball in RPM > > for Fedora use. > > > > * In third step - without any noise, just verified with fesco - > > https://pagure.io/fesco/issue/2907 - we stopped building JDK in fully > > integrated RPMs. Instead of this, normal RPMs BUildRequire portable > > RPMs and just unpack it, and repack it. > > > > Now last step is ahead - to build portable LTS JDKs 8,11,17 and 21 in > > oldest live Fedora, and repack everywhere. java-latest-openjdk, which > > contains latests STS JDK - currently 20, soon briefly 21 and a bit > > after 22... If we would built java-latest-openjdk in oldest live EPEL > > - epel8 now, we have verified, that such repacked JDKs works fine, > > however repack from epel seem to not be acceptable, thus > > ajva-latest-openjdk will be built twice - one in oldest live fedora, > > and once in oldest live epel. Build forme oldest possible epel will be > > repacked to that one or newer epels, and build from oldest live fedroa > > to all fedoras. > > > > === theoretical tagging solution === > > > > 1. request side tags for all releases > > 2. build the actual Java in the side tag for the oldest thing > > Could you use the real package name here. I think that will make it easier > to understand. You can still put 'actual Java' in parens or something. > > > 3. tag the result ot (2) to all side tags from (1) > > 4. waitrepo them > > 5. build the repacked java packages in all the side tags from (1) > > Same thing here, can you use the real package name. > > > 6. untag the result of (2) from all the side tags from (1) > > 7. ship bodhi updates from side tags OR retag the builds to candidate tags > > (and delete the side tags) > > > > The build from (2) will be eventually garbage collected. To prevent that, it > > might be re-tagged regularly. This is where releng might be able to help by > > creating a long lived tag to tag this into for preserving. > > > > Yes, we could make a 'fN-openjdk' tag and mark it protected... that part > > would be easy enough. > > > > I think this process could be improved if the side-tags in 1. are permanent side-tags, > and step 6 is skipped. That would make it easier to rebuild the packages from > step 5 if needed. > > Also, can you put a config file in the dist-git repos to tell fedpkg which target > to build against? I thought I remembered seeing that feature in the past. If so, > then you could configure the dist-gits for the repacked javas to automatically build > from those side-tags, which I think would be a lot easier for package maintainers and > may help make automated rebuilds possible. > > -Tom > > > > > > ==== including portable srpms in release (improving of step 6) ==== > > > > To include portable rpms in all live Fedoras is currently not > > possible. Best solution would be simply make and bodhi update of one > > portable rpm to all live fedoras. Bodhi is currenlty not capable to do > > so, issue was raised: > > https://github.com/fedora-infra/bodhi/issues/5387 investigating > > possibility to deliver single build as update to several releases. > > > > "..It's not possible ATM, it would require a heavy rewrite of the > > code, starting from the database structure (every build is now related > > to a single release)..." Maybe on long run..." > > > > On long run, if bodhi will allow this, that will be way to go. > > On short run, there are following options: > > a) ask releng to tag the portables directly > > - this needs manual approach of rare humans, thus no go unless > > strictly enforced by unpredicted conditions > > - this walks around whole testing repos. For portables tarballs, as > > nothing should depend on them, and are tested indirectly after repack, > > this should be technically ok, but is heavily discouraged in > > principle. > > b) build portable for all OSes, but do not ship them (don't do bodhi update) > > - this would probably work for all frontiers, only the real > > repacked JDK will be different > > - pros is, that we will be sure that portables builds on live fedoras > > - cons is, that the portable JDK will not be available by dnf install anyway > > c) build portable for all OSes, including bodhi update > > - pros is, that we will be sure that portables builds on live fedoras > > - another pros is that the portable JDK will be available by dnf > > install anyway > > - there may be clash during the build which will cause to repack > > wrong (newer, non certified) portables > > d) include SRPM_REBUILD.readme in srpm and generated > > PORTABLES_INSTALL.readme in RPMs, which will ideally at least contain: > > - instruction why you need portables > > - instruction how to find the portables > > - from SRPM_REBUILD.readme pointing to PORTABLES_INSTALL.readme > > - generated link to the koji, allowing to download the SRPM > > - generated link to the koji, allowing to download the binaries > > - generated instruction how to dnf install used portables > > > > I would currently vote for d). If there will be complains about broken > > SRPM rebuild, or need to install portables without hacking, then > > fall-back a, b or c via Change Proposal. > > Once Bodhi allows single build to be tagged to several release, I will > > move to that. > > > > == Feedback == > > > > > > == Benefit to Fedora == > > > > Java maintainers will finally have some free time... No kidding - > > maintenance and *certification* of so much supported JDKs on so much > > Fedora versions is brutal. By building once, and repack, we will > > regain cycles to continue support Fedora with all LTS and one STS JDK. > > > > If we fail to build once and repack everywhere, Java maintainers will > > most likely need to lower the number of JDKs in fedora to system one > > only. > > > > == Scope == > > * Proposal owners: Technically all JDKs (except 8, where some more > > tuning is needed, and EPEL for java-latest) are prepared, as they have > > a portable version, and RPMs just repack it. Except tuning up the JDK8 > > and EPEL for latest, scope owners are done. > > > > * Other developers: There will be needed significant support from RCM > > and maybe senior Fedora leadership to help to finish the build in > > oldest and enable to repack everywhere > > > > * '''Release engineering: [https://pagure.io/releng/issue/11438 > > #11438]''' There will be needed significant support from RCM, where > > I'm actually unsure what they will have to do to enable this. The mas > > rebuild will not be needed. > > > > * Policies and guidelines: AFAIK none (not needed for this Change) > > * Trademark approval: N/A (not needed for this Change) > > > > * Alignment with Community Initiatives: All supported JDKs will remain > > in Fedora in highest possible quality with full QA and certification, > > and its packagers will not lose their minds. Note that QA will still > > run on all live Fedoras, not only on the builder one. > > > > > > == Upgrade/compatibility impact == > > > > The change should be completely transparent to any user. > > > > > > == How To Test == > > > > `sudo dnf update/install "java*"` will install expected set of working packages. > > > > SRPM rebuild of both portables (which were built once) and of any rpms > > (from this freshly rebuild portbales) have to remain possible > > > > > > == User Experience == > > The change should be absolutely transparent to any user. > > > > > > == Dependencies == > > To finish this we will need heavy support from RCM, and maybe others. > > Although there are precedents with such pacakge, they all bites. From > > SW point of view, the dependece chain is `normal RPMs build requires > > portable RPMs` and thats all. > > > > > > == Contingency Plan == > > * Contingency mechanism: Even if It should be straight forward to > > revert back to building per OS, it '''may be impossible for current > > maintainers to save time''' for it. If this change is approved, we > > will be building '''4-5''' (jdk8,11,17,sts and 21) builds for all > > fedoras. If this change is not finished in time, we may '''need to > > orphan some of the JDKs'''. In better case, we will be able to keep > > living '''one LTS as system JDK, and java-latest-openjdk''' as future > > system JDK. That is 2*(3-5) builds (rawhide, (forked,), latest live, > > oldest live (oldest not yet dropped)). '''In worst case''', we may be > > able to maintain only java-latest-openjdk. On long run changing it to > > '''rolling system JDK,''' which are the expected 3-5 builds. > > * Contingency deadline: N/A > > * Blocks release? No. The change can be introduced even on the fly to > > live distributions. > > > > == Documentation == > > > > N/A (not a System Wide Change) > > > > == Release Notes == > > > > > > > > > _______________________________________________ > devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx > To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx > Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx > Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
