Once the portables will be built in f37, the repacked binary in f37 and in f38 (39...)will be identical. Rpms have some additional value - split to subpakcages, few symlinks due to system integration, system tzdata, system crypto binding and so on. On Thu, 29 Jun 2023 at 22:20, Tom Stellard <tstellar@xxxxxxxxxx> wrote: > > On 6/29/23 13:07, Jiri Vanek wrote: > > nn. You were right. There are going to be two separated packages. > > Portable, built once in oldest live, and "normal" which is going to > > repack them for all and shipp them. > > My apologise for typo in last second change: > > https://fedoraproject.org/w/index.php?title=Changes%2FBuildJdkOncePackEverywhere&type=revision&diff=681794&oldid=681791 > > > > narrowed. > > > > Ok, that makes sense now. > > My next question is what is the difference between the java-xy-openjdk-portable package > that is built on f37 and the repacked java-xy-openjdk package that is shipped on f38. > Are the bits inside the package exactly the same? > > -Tom > > > On Thu, 29 Jun 2023 at 21:16, Tom Stellard <tstellar@xxxxxxxxxx> wrote: > >> > >> On 6/29/23 11:06, Jiri Vanek wrote: > >>> Nope, xy stands for 1.8.0, 11, 17 and latest. It is enumerated several > >>> time in the proposal. Still the > >>> https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#theoretical_tagging_solution > >>> adjusted > >>> > >> > >> OK, I see. I thought there were going to be two different packages. java-xy-openjdk-portable > >> and java-xy-openjdk. Where java-xy-openjdk is the one that gets shipped in Fedora and > >> java-xy-openjdk-portable lives only in the side-tags. > >> > >> -Tom > >> > >>> Tahnx! > >>> > >>> On Thu, 29 Jun 2023 at 19:14, Tom Stellard <tstellar@xxxxxxxxxx> wrote: > >>>> > >>>> On 6/29/23 09:52, Jiri Vanek wrote: > >>>>> Hi Tom! > >>>>> > >>>>> Thanx a lot of for input. Although I did my bes with the tagging, it > >>>>> will be learning on the go. > >>>>> I had adapted https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#theoretical_tagging_solution > >>>>> as you suggested. It is great improvement. > >>>>> > >>>> > >>>> Thanks, this looks better. > >>>> > >>>> For step 5. should the first mention of java-xy-openjdk-portable actually > >>>> be java-xy-openjdk ? Same question for step 7. > >>>> > >>>> -Tom > >>>> > >>>>> Especially the config, I was not aware about. That woudl indeed help a lot. > >>>>> The usage of pernament tag is someging I have to learn, but is > >>>>> moreover necessary for proper srpm rebuil. > >>>>> > >>>>> TYVM! > >>>>> J. > >>>>> > >>>>> On Wed, 28 Jun 2023 at 21:31, Tom Stellard <tstellar@xxxxxxxxxx> wrote: > >>>>>> > >>>>>> On 6/26/23 09:21, Aoife Moloney wrote: > >>>>>>> https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#including_portable_srpms_in_release_(improving_of_step_6) > >>>>>>> > >>>>>>> This document represents a proposed Change. As part of the Changes > >>>>>>> process, proposals are publicly announced in order to receive > >>>>>>> community feedback. This proposal will only be implemented if approved > >>>>>>> by the Fedora Engineering Steering Committee. > >>>>>>> > >>>>>>> > >>>>>>> == Summary == > >>>>>>> > >>>>>>> This is the last step in > >>>>>>> https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs > >>>>>>> effort. JDKs in fedora are already static, and we repack portable > >>>>>>> tarballs into RPMs. Currently, the portable tarball is built for each > >>>>>>> Fedora and EPEL version. Goal here is to build each JDK > >>>>>>> (8,11,17,21,latest (20)) only once, in oldest live Fedora repack in > >>>>>>> all live Fedoras. If jdk is buitl in epel, it will be built in oldest > >>>>>>> possible epel and repacked in newer live epels. > >>>>>>> > >>>>>>> > >>>>>>> == Owner == > >>>>>>> * Name: [[User:jvanek| Jiri Vanek]] > >>>>>>> > >>>>>>> * Email: jvanek@xxxxxxxxxx > >>>>>>> > >>>>>>> > >>>>>>> == Detailed Description == > >>>>>>> > >>>>>>> As described in > >>>>>>> https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs ; > >>>>>>> during last year, packaging of JDKs had changed dramatically. As > >>>>>>> described in the same wiki page and in individual sub changes and > >>>>>>> devel threads, the primary reason for this is to lower maintenance and > >>>>>>> still keep Fedora Java friendly. > >>>>>>> > >>>>>>> * In the first system wide change, we have changed the JDKs to build > >>>>>>> properly as standalone, portable JDK - the way JDK is supposed to be > >>>>>>> built. I repeat, we spent ten years by patching JDK to become properly > >>>>>>> dynamic against system libs, and all patches went upstream, but this > >>>>>>> has become a fight which can not be won. > >>>>>>> > >>>>>>> * As a second step we introduced portable RPMs, which do not have any > >>>>>>> system integration, only build JDK and pack the final tarball in RPM > >>>>>>> for Fedora use. > >>>>>>> > >>>>>>> * In third step - without any noise, just verified with fesco - > >>>>>>> https://pagure.io/fesco/issue/2907 - we stopped building JDK in fully > >>>>>>> integrated RPMs. Instead of this, normal RPMs BUildRequire portable > >>>>>>> RPMs and just unpack it, and repack it. > >>>>>>> > >>>>>>> Now last step is ahead - to build portable LTS JDKs 8,11,17 and 21 in > >>>>>>> oldest live Fedora, and repack everywhere. java-latest-openjdk, which > >>>>>>> contains latests STS JDK - currently 20, soon briefly 21 and a bit > >>>>>>> after 22... If we would built java-latest-openjdk in oldest live EPEL > >>>>>>> - epel8 now, we have verified, that such repacked JDKs works fine, > >>>>>>> however repack from epel seem to not be acceptable, thus > >>>>>>> ajva-latest-openjdk will be built twice - one in oldest live fedora, > >>>>>>> and once in oldest live epel. Build forme oldest possible epel will be > >>>>>>> repacked to that one or newer epels, and build from oldest live fedroa > >>>>>>> to all fedoras. > >>>>>>> > >>>>>>> === theoretical tagging solution === > >>>>>>> > >>>>>>> 1. request side tags for all releases > >>>>>>> 2. build the actual Java in the side tag for the oldest thing > >>>>>> > >>>>>> Could you use the real package name here. I think that will make it easier > >>>>>> to understand. You can still put 'actual Java' in parens or something. > >>>>>> > >>>>>>> 3. tag the result ot (2) to all side tags from (1) > >>>>>>> 4. waitrepo them > >>>>>>> 5. build the repacked java packages in all the side tags from (1) > >>>>>> > >>>>>> Same thing here, can you use the real package name. > >>>>>> > >>>>>>> 6. untag the result of (2) from all the side tags from (1) > >>>>>>> 7. ship bodhi updates from side tags OR retag the builds to candidate tags > >>>>>>> (and delete the side tags) > >>>>>>> > >>>>>>> The build from (2) will be eventually garbage collected. To prevent that, it > >>>>>>> might be re-tagged regularly. This is where releng might be able to help by > >>>>>>> creating a long lived tag to tag this into for preserving. > >>>>>>> > >>>>>>> Yes, we could make a 'fN-openjdk' tag and mark it protected... that part > >>>>>>> would be easy enough. > >>>>>>> > >>>>>> > >>>>>> I think this process could be improved if the side-tags in 1. are permanent side-tags, > >>>>>> and step 6 is skipped. That would make it easier to rebuild the packages from > >>>>>> step 5 if needed. > >>>>>> > >>>>>> Also, can you put a config file in the dist-git repos to tell fedpkg which target > >>>>>> to build against? I thought I remembered seeing that feature in the past. If so, > >>>>>> then you could configure the dist-gits for the repacked javas to automatically build > >>>>>> from those side-tags, which I think would be a lot easier for package maintainers and > >>>>>> may help make automated rebuilds possible. > >>>>>> > >>>>>> -Tom > >>>>>> > >>>>>> > >>>>>>> > >>>>>>> ==== including portable srpms in release (improving of step 6) ==== > >>>>>>> > >>>>>>> To include portable rpms in all live Fedoras is currently not > >>>>>>> possible. Best solution would be simply make and bodhi update of one > >>>>>>> portable rpm to all live fedoras. Bodhi is currenlty not capable to do > >>>>>>> so, issue was raised: > >>>>>>> https://github.com/fedora-infra/bodhi/issues/5387 investigating > >>>>>>> possibility to deliver single build as update to several releases. > >>>>>>> > >>>>>>> "..It's not possible ATM, it would require a heavy rewrite of the > >>>>>>> code, starting from the database structure (every build is now related > >>>>>>> to a single release)..." Maybe on long run..." > >>>>>>> > >>>>>>> On long run, if bodhi will allow this, that will be way to go. > >>>>>>> On short run, there are following options: > >>>>>>> a) ask releng to tag the portables directly > >>>>>>> - this needs manual approach of rare humans, thus no go unless > >>>>>>> strictly enforced by unpredicted conditions > >>>>>>> - this walks around whole testing repos. For portables tarballs, as > >>>>>>> nothing should depend on them, and are tested indirectly after repack, > >>>>>>> this should be technically ok, but is heavily discouraged in > >>>>>>> principle. > >>>>>>> b) build portable for all OSes, but do not ship them (don't do bodhi update) > >>>>>>> - this would probably work for all frontiers, only the real > >>>>>>> repacked JDK will be different > >>>>>>> - pros is, that we will be sure that portables builds on live fedoras > >>>>>>> - cons is, that the portable JDK will not be available by dnf install anyway > >>>>>>> c) build portable for all OSes, including bodhi update > >>>>>>> - pros is, that we will be sure that portables builds on live fedoras > >>>>>>> - another pros is that the portable JDK will be available by dnf > >>>>>>> install anyway > >>>>>>> - there may be clash during the build which will cause to repack > >>>>>>> wrong (newer, non certified) portables > >>>>>>> d) include SRPM_REBUILD.readme in srpm and generated > >>>>>>> PORTABLES_INSTALL.readme in RPMs, which will ideally at least contain: > >>>>>>> - instruction why you need portables > >>>>>>> - instruction how to find the portables > >>>>>>> - from SRPM_REBUILD.readme pointing to PORTABLES_INSTALL.readme > >>>>>>> - generated link to the koji, allowing to download the SRPM > >>>>>>> - generated link to the koji, allowing to download the binaries > >>>>>>> - generated instruction how to dnf install used portables > >>>>>>> > >>>>>>> I would currently vote for d). If there will be complains about broken > >>>>>>> SRPM rebuild, or need to install portables without hacking, then > >>>>>>> fall-back a, b or c via Change Proposal. > >>>>>>> Once Bodhi allows single build to be tagged to several release, I will > >>>>>>> move to that. > >>>>>>> > >>>>>>> == Feedback == > >>>>>>> > >>>>>>> > >>>>>>> == Benefit to Fedora == > >>>>>>> > >>>>>>> Java maintainers will finally have some free time... No kidding - > >>>>>>> maintenance and *certification* of so much supported JDKs on so much > >>>>>>> Fedora versions is brutal. By building once, and repack, we will > >>>>>>> regain cycles to continue support Fedora with all LTS and one STS JDK. > >>>>>>> > >>>>>>> If we fail to build once and repack everywhere, Java maintainers will > >>>>>>> most likely need to lower the number of JDKs in fedora to system one > >>>>>>> only. > >>>>>>> > >>>>>>> == Scope == > >>>>>>> * Proposal owners: Technically all JDKs (except 8, where some more > >>>>>>> tuning is needed, and EPEL for java-latest) are prepared, as they have > >>>>>>> a portable version, and RPMs just repack it. Except tuning up the JDK8 > >>>>>>> and EPEL for latest, scope owners are done. > >>>>>>> > >>>>>>> * Other developers: There will be needed significant support from RCM > >>>>>>> and maybe senior Fedora leadership to help to finish the build in > >>>>>>> oldest and enable to repack everywhere > >>>>>>> > >>>>>>> * '''Release engineering: [https://pagure.io/releng/issue/11438 > >>>>>>> #11438]''' There will be needed significant support from RCM, where > >>>>>>> I'm actually unsure what they will have to do to enable this. The mas > >>>>>>> rebuild will not be needed. > >>>>>>> > >>>>>>> * Policies and guidelines: AFAIK none (not needed for this Change) > >>>>>>> * Trademark approval: N/A (not needed for this Change) > >>>>>>> > >>>>>>> * Alignment with Community Initiatives: All supported JDKs will remain > >>>>>>> in Fedora in highest possible quality with full QA and certification, > >>>>>>> and its packagers will not lose their minds. Note that QA will still > >>>>>>> run on all live Fedoras, not only on the builder one. > >>>>>>> > >>>>>>> > >>>>>>> == Upgrade/compatibility impact == > >>>>>>> > >>>>>>> The change should be completely transparent to any user. > >>>>>>> > >>>>>>> > >>>>>>> == How To Test == > >>>>>>> > >>>>>>> `sudo dnf update/install "java*"` will install expected set of working packages. > >>>>>>> > >>>>>>> SRPM rebuild of both portables (which were built once) and of any rpms > >>>>>>> (from this freshly rebuild portbales) have to remain possible > >>>>>>> > >>>>>>> > >>>>>>> == User Experience == > >>>>>>> The change should be absolutely transparent to any user. > >>>>>>> > >>>>>>> > >>>>>>> == Dependencies == > >>>>>>> To finish this we will need heavy support from RCM, and maybe others. > >>>>>>> Although there are precedents with such pacakge, they all bites. From > >>>>>>> SW point of view, the dependece chain is `normal RPMs build requires > >>>>>>> portable RPMs` and thats all. > >>>>>>> > >>>>>>> > >>>>>>> == Contingency Plan == > >>>>>>> * Contingency mechanism: Even if It should be straight forward to > >>>>>>> revert back to building per OS, it '''may be impossible for current > >>>>>>> maintainers to save time''' for it. If this change is approved, we > >>>>>>> will be building '''4-5''' (jdk8,11,17,sts and 21) builds for all > >>>>>>> fedoras. If this change is not finished in time, we may '''need to > >>>>>>> orphan some of the JDKs'''. In better case, we will be able to keep > >>>>>>> living '''one LTS as system JDK, and java-latest-openjdk''' as future > >>>>>>> system JDK. That is 2*(3-5) builds (rawhide, (forked,), latest live, > >>>>>>> oldest live (oldest not yet dropped)). '''In worst case''', we may be > >>>>>>> able to maintain only java-latest-openjdk. On long run changing it to > >>>>>>> '''rolling system JDK,''' which are the expected 3-5 builds. > >>>>>>> * Contingency deadline: N/A > >>>>>>> * Blocks release? No. The change can be introduced even on the fly to > >>>>>>> live distributions. > >>>>>>> > >>>>>>> == Documentation == > >>>>>>> > >>>>>>> N/A (not a System Wide Change) > >>>>>>> > >>>>>>> == Release Notes == > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>> _______________________________________________ > >>>>>> devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx > >>>>>> To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx > >>>>>> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > >>>>>> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > >>>>>> List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx > >>>>>> Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue > >>>>> > >>>> > >>> > >> > > > _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
