Re: F39 Change Proposal: Build JDKs once, repack everywhere (System-Wide) - Proposal Addendum

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Nope, xy stands for 1.8.0, 11, 17 and latest. It is enumerated several
time in the proposal. Still the
https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#theoretical_tagging_solution
adjusted

Tahnx!

On Thu, 29 Jun 2023 at 19:14, Tom Stellard <tstellar@xxxxxxxxxx> wrote:
>
> On 6/29/23 09:52, Jiri Vanek wrote:
> > Hi Tom!
> >
> > Thanx a lot of for input. Although I did my bes with the tagging, it
> > will be learning on the go.
> > I had adapted https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#theoretical_tagging_solution
> > as you suggested. It is great improvement.
> >
>
> Thanks, this looks better.
>
> For step 5. should the first mention of java-xy-openjdk-portable actually
> be java-xy-openjdk ?  Same question for step 7.
>
> -Tom
>
> > Especially the config, I was not aware about. That woudl indeed help a lot.
> > The usage of pernament tag is someging I have to learn, but is
> > moreover necessary for proper srpm rebuil.
> >
> > TYVM!
> >   J.
> >
> > On Wed, 28 Jun 2023 at 21:31, Tom Stellard <tstellar@xxxxxxxxxx> wrote:
> >>
> >> On 6/26/23 09:21, Aoife Moloney wrote:
> >>> https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#including_portable_srpms_in_release_(improving_of_step_6)
> >>>
> >>> This document represents a proposed Change. As part of the Changes
> >>> process, proposals are publicly announced in order to receive
> >>> community feedback. This proposal will only be implemented if approved
> >>> by the Fedora Engineering Steering Committee.
> >>>
> >>>
> >>> == Summary ==
> >>>
> >>> This is the last step in
> >>> https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs
> >>> effort. JDKs in fedora are already static, and we repack portable
> >>> tarballs into RPMs. Currently, the portable tarball is built for each
> >>> Fedora and EPEL version. Goal here is to build each JDK
> >>> (8,11,17,21,latest (20)) only once, in oldest live Fedora repack in
> >>> all live Fedoras. If jdk is buitl in epel, it will be built in oldest
> >>> possible epel  and repacked in newer live epels.
> >>>
> >>>
> >>> == Owner ==
> >>> * Name: [[User:jvanek| Jiri Vanek]]
> >>>
> >>> * Email: jvanek@xxxxxxxxxx
> >>>
> >>>
> >>> == Detailed Description ==
> >>>
> >>> As described in
> >>> https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs ;
> >>> during last year, packaging of JDKs had changed dramatically. As
> >>> described in the same wiki page and in individual sub changes and
> >>> devel threads, the primary reason for this is to lower maintenance and
> >>> still keep Fedora Java friendly.
> >>>
> >>> * In the first system wide change, we have changed the JDKs to build
> >>> properly as standalone, portable JDK - the way JDK is supposed to be
> >>> built. I repeat, we spent ten years by patching JDK to become properly
> >>> dynamic against system libs, and all patches went upstream, but this
> >>> has become a fight which can not be won.
> >>>
> >>> * As a second step we introduced portable RPMs, which do not have any
> >>> system integration, only build JDK and pack the final tarball in RPM
> >>> for Fedora use.
> >>>
> >>> * In third step - without any noise, just verified with fesco -
> >>> https://pagure.io/fesco/issue/2907 - we stopped building JDK in fully
> >>> integrated RPMs. Instead of this, normal RPMs BUildRequire portable
> >>> RPMs and just unpack it, and repack it.
> >>>
> >>> Now last step is ahead - to build portable LTS JDKs 8,11,17 and 21 in
> >>> oldest live Fedora, and repack everywhere. java-latest-openjdk, which
> >>> contains latests STS JDK - currently 20, soon briefly 21 and a bit
> >>> after 22... If we would built java-latest-openjdk in  oldest live EPEL
> >>> - epel8 now, we have verified, that such repacked JDKs works fine,
> >>> however repack from epel seem to not be acceptable, thus
> >>> ajva-latest-openjdk will be built twice - one in oldest live fedora,
> >>> and once in oldest live epel. Build forme oldest possible epel will be
> >>> repacked to that one or newer epels, and build from oldest live fedroa
> >>> to all fedoras.
> >>>
> >>> === theoretical tagging solution ===
> >>>
> >>>     1. request side tags for all releases
> >>>     2. build the actual Java in the side tag for the oldest thing
> >>
> >> Could you use the real package name here.  I think that will make it easier
> >> to understand.  You can still put 'actual Java' in parens or something.
> >>
> >>>     3. tag the result ot (2) to all side tags from (1)
> >>>     4. waitrepo them
> >>>     5. build the repacked java packages in all the side tags from (1)
> >>
> >> Same thing here, can you use the real package name.
> >>
> >>>     6. untag the result of (2) from all the side tags from (1)
> >>>     7. ship bodhi updates from side tags OR retag the builds to candidate tags
> >>>        (and delete the side tags)
> >>>
> >>> The build from (2) will be eventually garbage collected. To prevent that, it
> >>> might be re-tagged regularly. This is where releng might be able to help by
> >>> creating a long lived tag to tag this into for preserving.
> >>>
> >>> Yes, we could make a 'fN-openjdk' tag and mark it protected... that part
> >>> would be easy enough.
> >>>
> >>
> >> I think this process could be improved if the side-tags in 1. are permanent side-tags,
> >> and step 6 is skipped.  That would make it easier to rebuild the packages from
> >> step 5 if needed.
> >>
> >> Also, can you put a config file in the dist-git repos to tell fedpkg which target
> >> to build against?  I thought I remembered seeing that feature in the past.  If so,
> >> then you could configure the dist-gits for the repacked javas to automatically build
> >> from those side-tags, which I think would be a lot easier for package maintainers and
> >> may help make automated rebuilds possible.
> >>
> >> -Tom
> >>
> >>
> >>>
> >>> ==== including portable srpms in release (improving of step 6) ====
> >>>
> >>> To include portable  rpms in all live Fedoras is currently not
> >>> possible. Best solution would be simply make and bodhi update of one
> >>> portable rpm to all live fedoras. Bodhi is currenlty not capable to do
> >>> so, issue was raised:
> >>> https://github.com/fedora-infra/bodhi/issues/5387 investigating
> >>> possibility to deliver single build as update to several releases.
> >>>
> >>> "..It's not possible ATM, it would require a heavy rewrite of the
> >>> code, starting from the database structure (every build is now related
> >>> to a single release)..." Maybe on long run..."
> >>>
> >>> On long run, if bodhi will allow this, that will be way to go.
> >>> On short run, there are following options:
> >>>    a) ask releng to tag the portables directly
> >>>      - this needs manual approach of rare humans, thus no go unless
> >>> strictly enforced by unpredicted conditions
> >>>      - this walks around whole testing repos. For portables tarballs, as
> >>> nothing should depend on them, and are tested indirectly after repack,
> >>> this should be technically ok, but is heavily discouraged in
> >>> principle.
> >>>    b) build portable for all OSes, but do not ship them (don't do bodhi update)
> >>>      - this would probably work for all frontiers, only the real
> >>> repacked JDK will be different
> >>>      - pros is, that we will be sure that portables builds on live fedoras
> >>>      - cons is, that the portable JDK will not be available by dnf install anyway
> >>>    c) build portable for all OSes, including bodhi update
> >>>      - pros is, that we will be sure that portables builds on live fedoras
> >>>      - another pros is that the portable JDK will be available by dnf
> >>> install anyway
> >>>      - there may be clash during the build  which will cause to repack
> >>> wrong (newer, non certified) portables
> >>>    d) include SRPM_REBUILD.readme in srpm and generated
> >>> PORTABLES_INSTALL.readme in RPMs, which will ideally at least contain:
> >>>      - instruction why you need portables
> >>>      - instruction how to find the portables
> >>>      - from SRPM_REBUILD.readme pointing to PORTABLES_INSTALL.readme
> >>>      - generated link to the  koji, allowing to download the SRPM
> >>>      - generated link to the  koji, allowing to download the binaries
> >>>      - generated instruction how to dnf install used portables
> >>>
> >>> I would currently vote for d). If there will be complains about broken
> >>> SRPM rebuild, or need to install portables without hacking, then
> >>> fall-back  a, b or c via Change Proposal.
> >>> Once Bodhi allows single build to be tagged to several release, I will
> >>> move to that.
> >>>
> >>> == Feedback ==
> >>>
> >>>
> >>> == Benefit to Fedora ==
> >>>
> >>> Java maintainers will finally have some free time... No kidding -
> >>> maintenance and *certification* of so much supported JDKs on so much
> >>> Fedora versions is brutal.  By building once, and repack, we will
> >>> regain cycles to continue support Fedora with all LTS and one STS JDK.
> >>>
> >>> If we fail to build once and repack everywhere, Java maintainers will
> >>> most likely need to lower the number of JDKs in fedora to system one
> >>> only.
> >>>
> >>> == Scope ==
> >>> * Proposal owners: Technically all JDKs (except 8, where some more
> >>> tuning is needed, and EPEL for java-latest) are prepared, as they have
> >>> a portable version, and RPMs just repack it. Except tuning up the JDK8
> >>> and EPEL for latest, scope owners are done.
> >>>
> >>> * Other developers: There will be needed significant support from RCM
> >>> and maybe senior Fedora leadership to help to finish the build in
> >>> oldest and enable to repack everywhere
> >>>
> >>> * '''Release engineering: [https://pagure.io/releng/issue/11438
> >>> #11438]'''  There will be needed significant support from RCM, where
> >>> I'm actually unsure what they will have to do to enable this. The mas
> >>> rebuild will not be needed.
> >>>
> >>> * Policies and guidelines: AFAIK none (not needed for this Change)
> >>> * Trademark approval: N/A (not needed for this Change)
> >>>
> >>> * Alignment with Community Initiatives: All supported JDKs will remain
> >>> in Fedora in highest possible quality with full QA and certification,
> >>> and its packagers will not lose their minds. Note that QA will still
> >>> run on all live Fedoras, not only on the builder one.
> >>>
> >>>
> >>> == Upgrade/compatibility impact ==
> >>>
> >>> The change should be completely transparent to any user.
> >>>
> >>>
> >>> == How To Test ==
> >>>
> >>> `sudo dnf update/install "java*"` will install expected set of working packages.
> >>>
> >>> SRPM rebuild of both portables (which were built once) and of any rpms
> >>> (from this freshly rebuild portbales) have to remain possible
> >>>
> >>>
> >>> == User Experience ==
> >>> The change should be absolutely transparent to any user.
> >>>
> >>>
> >>> == Dependencies ==
> >>> To finish this we will need heavy support from RCM, and maybe others.
> >>> Although there are precedents with such pacakge, they all bites. From
> >>> SW point of view, the dependece chain is `normal RPMs build requires
> >>> portable RPMs` and thats all.
> >>>
> >>>
> >>> == Contingency Plan ==
> >>> * Contingency mechanism: Even if It should be straight forward to
> >>> revert back to building per OS, it '''may be impossible for current
> >>> maintainers to save time''' for it.  If this change is approved, we
> >>> will be building '''4-5''' (jdk8,11,17,sts and 21) builds for all
> >>> fedoras. If this change is not finished in time, we may '''need to
> >>> orphan some of the JDKs'''. In better case, we will be able to keep
> >>> living '''one LTS as system JDK, and java-latest-openjdk''' as future
> >>> system JDK. That is 2*(3-5) builds (rawhide, (forked,), latest live,
> >>> oldest live (oldest not yet dropped)).  '''In worst case''', we may be
> >>> able to maintain only java-latest-openjdk. On long run changing it to
> >>> '''rolling system JDK,''' which are the expected 3-5 builds.
> >>> * Contingency deadline: N/A
> >>> * Blocks release?  No. The change can be introduced even on the fly to
> >>> live distributions.
> >>>
> >>> == Documentation ==
> >>>
> >>> N/A (not a System Wide Change)
> >>>
> >>> == Release Notes ==
> >>>
> >>>
> >>>
> >>>
> >> _______________________________________________
> >> devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
> >> To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
> >> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> >> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> >> List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
> >> Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
> >
>
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue




[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Fedora Announce]     [Fedora Users]     [Fedora Kernel]     [Fedora Testing]     [Fedora Formulas]     [Fedora PHP Devel]     [Kernel Development]     [Fedora Legacy]     [Fedora Maintainers]     [Fedora Desktop]     [PAM]     [Red Hat Development]     [Gimp]     [Yosemite News]

  Powered by Linux