Re: F37 proposal: Build all JDKs in Fedora against in-tree libraries and with static stdc++lib (System-Wide Change proposal)

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Jiri Vanek wrote:
> see the clarifications of most concerns:

Unfortunately, your mail does not clarify all that much for me. I actually 
see several contradictions, e.g.:

>  > Are people really installing OpenJDK RPM packages, taking the
>  > "/usr/bin/java" binary, and putting it onto some other system?
> No, no and no:)

vs.

> To call build of OpenJDK java, each binary ahve to pas TCK(as mentioned in
> thread). And you have to do it on each binary you produce. So with dynamic
> lining in fedoras, I have to certifie 12+ builds. And update once they
> pass (with small exception in rawhide). With portable build, I can build
> once, TCK it eg on super custom opensuse, and still do update (from this
> binary) to all Fedoras.

so "putting [the binary] onto some other system", "eg on super custom 
opensuse", is exactly what you want to be able to do.

> Not necessarily. In small project, sure, bundled libraries will get
> rotten, but project like OpenJDK, where 99% of its builds uses the in tree
> copies, can not allow itself to have security holes in them. It already
> happened in past that we have switched to the in-tree library because the
> CVE there was already fixed, and moved back to system library once the fix
> landed to live Fedora.

There will necessarily be a delay between a security hole being fixed in a 
commonly used library such as zlib or freetype and the fix making it into an 
OpenJDK release.

And in the case you describe (where the system version is not fixed yet when 
OpenJDK upstream is), switching to the bundled version is the wrong thing to 
do, you need to get the security update to the Fedora library pushed out as 
quickly as possible, so everything benefits from the security fix. (Maybe 
you or another OpenJDK maintainer needs provenpackager privileges for that.)

>  > Applications linking against libraries SHOULD link against shared
>  > libraries not static versions.
>  > https://docs.fedoraproject.org/en-US/packaging-guidelines/#packaging-stat...
> 
> right, should. So strictly from law of guidelines, there is no rule to
> stop this change. The bundling was always bad, but is necessary evil.

SHOULD still means you need to justify why you want to do otherwise. I do 
not see a valid reason in this case.

> See eg rsync - the bundled zlib is there because it is technically not
> possible to use dynamic one.

In the rsync case, it is actually also disputed whether this is really 
necessary (I doubt that there is really no better way to solve their 
problem), but at least there is a concrete issue there, which (IIRC) is wire 
compatibility of checksums of compressed blocks.

> Java is in the border - yes, you can use the dynamic libraries (and it
> really took several excellent engineers a decade to manage), but it have 
> pros and cons. Unluckily, the negatives are multiplying for years.
> Although we are not happy with the change, it must be done if JDKs should
> remain maintained and healthy in Fedora. s

I have not seen any issues with the dynamic builds over years of using 
Fedora OpenJDK RPMs for production use.

>  > And upstream only incorporates security fixes once per quarter
> 
> Right, but downstream works well too. If such vulnerability occurs, be
> sure we will patch RPMs asap, and also upstream project - maybe without
> release - will react to.

And that is a lot of work you are trying to commit to, in the same mail 
where you are stating that you already have a hard time keeping up with the 
workload in the current state. I do not see how you can realistically get 
this done without significant delays for security fixes in the bundled 
libraries, especially if you are going to wait for TCK results for each and 
every security backport to a bundled library.

>  > In this case upstream might actually get there first because this CVE
>  > is not yet fixed in Fedora
> 
> And it already happeed in past, that jdk swithced to bundled version for a
> while, and back to dynamic once the library was fixed in live Fedora.

And as I wrote above, that is entirely the wrong way to go at it.

>  > This sounds fascinating. Can anyone share details about this? On the
> I'm  aware of some codecs, which are built in Fedora, then the binary is
> sent to .. cisco(?), and  if passed, they are repacked into all live
> fedoras.

OpenH264 is actually *not* part of Fedora, as Red Hat is not allowed to ship 
it under the patent license, only Cisco is. So it gets built on the Fedora 
Koji in an unpublished tag, shipped from there to Cisco, and Cisco releases 
it in a third-party repository that is enabled by default in the Fedora 
repository configuration (but can be disabled by the user, e.g., I have it 
disabled because the FFmpeg H.264 decoder and the x264 encoder, both from 
RPM Fusion, are simply the better H.264 implementations). That is a very 
special arrangement that is due to patent issues.

I do not see how OpenJDK qualifies for such a special arrangement, and even 
if it does, what you want is exactly the opposite of what OpenH264 is doing: 
You want to get a build *into* Fedora repositories that is not built in the 
respective Koji buildroot (it may be built in Koji, but you want to build it 
once for all Fedora releases, so not in the release's buildroot), whereas 
OpenH264 is actually built *in* the release's Koji buildroot and then 
shipped *elsewhere*.

> As super cornercase of this may be packing of some firmwares as
> binary blobs.

This is an exception specific to firmware (it is treated as content, not 
code), which explicitly does *not* apply to anything running on the CPU in 
kernel space or user space.

>  > versions in an RPM sounds like it would violate all the guidelines
>  > under $foo to meet this threshold and claim it needs to avoid
>  > rebuilding for certification?
> 
> Right, you need fesco exception.

And I sure hope it would not be granted, as I do not see a valid rationale 
for such a blatant guideline violation at all.

>  > I'm very confused on why this reduces certification burden. In our 
>  > crypto libraries this is exactly the kind of behavior we would *NOT* 
>  > want packages to do because it increases our certification and support 
>  > burden.
> I believe this are two things.
> First - our burden. We ahve to certify each binary. This is quite long and
> lenghty process. Onl once it is certified, we can release it (with small
> unwritten exception in rawhide)
> So with repacked portabel build as described  in
> https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs will
> allow us to certify once (and even stop hoping for that small exception in
> rawhide). Considering 4 jdks in  3 maintianed fedoras, that is incredible
> work. If it reduced to 4 we will manage again. Second - I can understand
> that you certify your libray, but if others bundle it, then it is theirs
> choice, and they are on thier own, arent they? Also you highlight crypto
> lib, we are not going to embed any crypto library. Crypto is loaded in
> runtime, and build time have no idea about future system in this case. Can
> you please elaborate more if still needed?

Andrew Hughes says you actually do not have to certify each binary, so 
logically one of you must be mistaken.

>  > I would rather have our shared maintenance and evolution of font stuff
>  > be reused in Java too...
> 
> This is holy grail we have been pursuing for last 10 years.  But now we
> gave up. To keep java alive in Fedora, we have to take this one step back.

At that point, I do not see the value of keeping Java alive in Fedora at 
all. Better work with Adoptium to get their builds into an RPM repository 
(hosted by them), if this kind of vanilla all-bundled builds are what you 
want.

And maybe if you orphan the Fedora RPM, someone might pick it up. As a 
Fedora packager and OpenJDK user, I might even sign up to comaintain, though 
the time I can spend on it is limited.

I think the whole Java SIG needs to be restarted from scratch anyway, as 
there is very little left from the Java ecosystem that was once packaged in 
Fedora. OpenJDK is almost the last piece that is remaining.

> There alre already prcednets, and even worse things - like packing blobish
> binary driver, are here.

Fedora does *not* package binary driver blobs. We do package binary firmware 
blobs. The distinction between a driver and a firmware is that the driver 
runs on the CPU, the firmware runs on the peripheral. (Then there is the 
early boot firmware that runs before the OS (UEFI/BIOS) or gets loaded at 
early boot to fix CPU bugs (CPU microcode), those are special cases.) 
OpenJDK is *not* firmware. (And for that matter, it is not a driver either.)

> As described above, keeping dynamic jdk proeprly working requires huge
> maintainance. In addition, jdks are expected to be static (only
> distribution rpms do differently, and thats why developers often unpack
> 3rd party static jdk to develop on)

I have been developing in Java for a living for more than 8 years now. I 
have never needed any other JDK than the OpenJDK RPMs from the Fedora 
repository. So I have no idea what developers you are talking about.

> and have futures like generating standalone binary native images and so,
> which are impossible with dynamic linking. Future of java looks pointing
> pretty stright forward to such changes, so we have to move to.

I take it that those "standalone binary native images" really just bundle a 
static java binary with the bytecode into an executable?

And what if I want to build the code on Fedora for a customer running 
Windows, can I do that with this feature? It does not sound like I can. 
Where I work, some of us developers run GNU/Linux (several different 
distributions, I run Fedora) and some run Windows, the customers typically 
all run Windows (though of course we could easily support customers running 
GNU/Linux, as we do have all the JNI libraries and wrapper scripts in place 
for us GNU/Linux-using developers). The ability to build once and run 
anywhere is one of the reasons we are using Java to begin with.

In any case, we have never used this feature at work so far. I was not even 
aware that it exists at all. We deliver Java code in 2 ways: 1. as JARs in a 
ZIP or tarball (for anything that needs to run on the customer's 
infrastructure), or 2. as a web service in Tomcat hosted on our servers 
(which run Debian and the OpenJDK and Tomcat from the standard Debian 
repositories). For JARs we deliver to our customers, we require a systemwide 
JRE installation (and we can point them to the Adoptium Windows binaries if 
they need a specific recommendation – as I wrote above, the customers 
typically run Windows).

> Exactly. With one addition - the jdk will be indeed portable, you should
> be able to bring it to most of the system,and it should work. Still glibc
> limitations remains here. To stand on the safe ground, "usual" non
> distribution providers of openjdk simply builds on oldest possible
> supported system.

Well yes, that is what I do for the JNI shared objects we ship, too (and 
mock is a great tool for that). But if you want such a build, then use a 
non-distro OpenJDK build, not a distro one.

>  > This is so bad. The final death of Java on Fedora.
> 
> Actually just a opposite. This is future of java and without it java in
> fedora my diminish and fade away.

Has it not already? OpenJDK is essentially the last piece that is left and 
your proposal would make that package entirely uninteresting and pointless 
too (by making it no different from Adoptium Temurin and the like).

> I personally really do not like this change, and I agree with all rebukes
> taken here. But current OpenJDK maintainers (which are the same for last
> decade) do not see other way. If it will really go bad, we will withdraw
> and continue fighting.

Maybe at this point withdrawing would be the most honest thing to do? IMHO, 
we need either new people willing to bring the Java ecosystem (and that 
includes Maven, Eclipse and its platform, NetBeans and its platform, etc.) 
back to Fedora, following Fedora rules and guidelines, and not constantly 
trying to work around them (see the module-only packages in the past, and 
now this proposal and its followups), or we need to admit failure and just 
drop Java from Fedora altogether (and focus on getting a third-party 
repository delivering Adoptium Temurin binaries up ASAP as a replacement / 
upgrade path).

> Where you are right with the interpreters/jits not being exempt, I'm
> afraid yo do not see current flow of java - to self contained native
> images, and to share OpenJDK marketpalce. Even if wesomhow will be able to
> pathc jdk to live with
> system libraries, which is already close to  impossible, and drop all
> features which allows generate portable images, the problem with the
> certification will persists. It isnot in human powers to maintain 4 JDKS
> in 3 Fedoras.

As I wrote above, if it is not feasible (with the available manpower) to 
maintain OpenJDK properly in Fedora, then it needs to be dropped.

>  > And this plan is entirely unacceptable. It is just plain not allowed in
>  >  Fedora to ship prebuilt binary blobs (even if they are built by Fedora
>  >  developers)
> 
> This is wrong.  The JDK will be always build from sources in koji. The
> main reduce of load will be that webuilt once in koji, in oldest Fedora,
> and repack to newer.

Building once in Koji, extracting it, and then resending the resulting blob 
to Koji for all releases is not how the Fedora workflow works and still 
violates the "no prebuilt binaries" rule, even if the binary technically has 
been built in some Koji buildroot at some point.

At most, you may be able to build in Koji and then get that exact build 
*tagged* for the different releases (so you do not send blobs to Koji, you 
just tell Koji that the build output tagged, say, f35-updates, should also 
get tagged with f36-updates and f37), but I believe this is a kind of 
hackery we also stopped doing long ago.

> In additon there are many excludes in various binary drivers.

As I explained above, Fedora does not ship binary drivers.

I am only aware of exceptions of this kind having been granted (in the past) 
for huge data files. Maybe also firmware. But by definition, these are not 
CPU executables and hence building them on different Fedora releases will 
(or at least should) produce identical results. This is not the case for 
OpenJDK executables.

Also, those have been handled using the tagging approach I have described 
above, so that really the exact same RPMs are used on all releases, and then 
the files had even been hardlinked on the master mirror and on mirrors that 
support it. Otherwise, there would have been no benefit from doing this at 
all for those huge data files.

And as far as I can tell, we also no longer do this. E.g., the huge 0ad-data 
is built separately for each release:
https://koji.fedoraproject.org/koji/packageinfo?packageID=14763

> It is not as simple. You are right that to call it java yoou must pass
> tck. But unluckily even IcedTea had to pass TCK. Maybe it is possible to
> patch out all "java" from it, but I doubt jdk will remain working and
> usefull,.

(Disclaimer: The following is not legal advice, I am not a lawyer.) I do not 
see why you would have to rename, e.g., the java and javac executables or 
the java.* and javax.* packages. To my knowledge, legal reviews have ruled 
several times in similar cases that this kind of executable or package names 
is part of an API and not trademarkable. (I do not know whether this has 
ever been tested in court.) The Oracle proprietary license places several 
restrictions on what you can do with the java.* and javax.* packages, but we 
use OpenJDK under the GPL, not under that license, so I do not see why those 
non-free license restrictions would affect us in any way. So I think it 
would be enough to just rename, e.g., java-11-openjdk to openjdk-11 (and you 
could still Obsolete and Provide java-11-openjdk for an upgrade path) and 
remove the word "Java" from Summary and Description.

> The simple rename is not so easy. I "m afraid that striping all "java"
> from bnaries and thus from sources is maybe possbel to be done, but  human
> impossible to maintian, and will leave JDK mostly not working, and for
> sure useless and incomaptible with other javas.

(Disclaimer: The following is not legal advice, I am not a lawyer.) You do 
not need to strip any mentions of Java that are required for 
interoperability (see, e.g., Sega v. Accolade). Nor do you need to strip any 
uses of the trademark that are not user-visible at all.

>  > If current maintainers can't continue maintaining the well-packaged
>  > OpenJDK, I think it's time to retire it. It would be better
> Yes, we can no longer maintian it. And I must contradict you - Seeing all
> the dependencies, we really need them all. Loosing any  one, wiould kill
> java ecosystemin Fedora.

What is still left from it other than OpenJDK to begin with? But yes, of 
course, retiring OpenJDK would also require retiring all other remaining 
Java packages. That is obvious.

>  >> JDKs from other vendors(Amazon, Azul, Oracle, etc.) are built in
>  >> exactly this way. W
>  > Because they build it for all available GNU/Linux distributions. We
>  > shouldn't focus on bad practices.
> 
> Well and there are many reasons to do so, as jdk is designed to be like
> this.  for 10 years we tried to go against the main stream, and we managed
> a lot. But to keep peace and compatibility and proper developer's support,
> we have to move with mian stream now. The divergences we keep in rpms are
> right now blocking devloeprs to use system jdsk as proper JDKs and are
> enforcing them to download Amazon, Azul, Oracle, etc. blobs... To keep
> Fedora competitive, this is currenlty necessary step.

Which developers? The divergences are sure not blocking me.

If some Java developers (or even end users) want the blobs, then they should 
just use the blobs. Nothing is stopping them. I see no value whatsoever in 
providing the same kind of static all-bundled builds disguised as native 
Fedora packages.

        Kevin Kofler
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure




[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Fedora Announce]     [Fedora Users]     [Fedora Kernel]     [Fedora Testing]     [Fedora Formulas]     [Fedora PHP Devel]     [Kernel Development]     [Fedora Legacy]     [Fedora Maintainers]     [Fedora Desktop]     [PAM]     [Red Hat Development]     [Gimp]     [Yosemite News]

  Powered by Linux