Hi Toshio,
Toshio Kuratomi wrote:
On Thu, 2007-01-11 at 17:49 -0500, Fernando Nasser wrote:
1) Problem Space
"Many Java programs are first packaged by the [WWW] jpackage project. In
essence, the Fedora packages have two upstreams: The upstream project
itself and the jpackage project's spec file and patches. Fedora imports
a package from jpackage and makes changes to enable native AOT
compilation via gcj and bugfixes that have not been merged into either
of the upstreams."
> Well, The last sentence is no longer true. The AOT compilation bits
were propagated upstream, so all the Java team has to do now is to
import the upstram package as is and build. We still need to make any
changes to the code to circumvent GCJ shortcomes, but these are so rare
nowadays that almost all packages build without any changes at all.
Sounds good.
> As a consequence, fixes have been made on the usptream version, which
is them re-imported, and we've been also reimporting when fixes are
applied upstream comming from the other JPackage-based distros or from
direct JPackage users feedback.
Sounds like a good deal.
"Presently, Core's jpackage-based packages have reflected the package
origin within the package Release: 1jpp_2fc In this scheme, the 1jpp is
the upstream jpackage release, _ is a separator, and 2fc is the Fedora
release. (Note, this scheme is not currently as defined as the Fedora
Naming Guidelines. So there are some practices above the base standard
that would need to change.) This scheme has two expressed goals:"
> OK, here we have made much progress:
a. The "_" character has been removed and replaced by a '.'
b. The "fc" string has been removed
c. All FC6 packages have been rebuilt to comply with a and b above
Reading this it looks like the packages are now namedSo
name-numericversion-1jpp upstream and name-numericversion-1jpp.2 by
Fedora. Is that correct?
Yes, we were able to do this without breaking any of our scripts or
processes and it is much more clean (no extraneous or superfluous
characters).
d. An initial agreement has been reached to also compatibilize the
pre-release cases, with JPackage adopting a Fedora-like order.
Do you have a link to this documentation? I found this:
http://www.jpackage.org/cgi-bin/viewvc.cgi/*checkout*/src/jpackage-utils/doc/jpackage-1.5-policy.xhtml?root=jpackage&content-type=text%2Fhtml
Which doesn't cover versioning and this:
http://www.jpackage.org/jpppolicy.php
which says it's the old version and indeed, has the old versioning
information.
Yes, you are right. As you see, it says 1.5 there. That was for JPP
1.5, and JPackage is releasing JPP 1.7 soon (it is release candidate now).
The idea is that as soon as we figure out how we are going to handle the
imports into Fedora a "Guidelines" link will be added with updated
recommendations. There is a couple of things that must be known first:
(a) what is the decision on that thread on Epoch and how that affects
the Java packages (they currently always specify the Epoch) (b) the
exact syntax for the pre-release tags so that is compatible with Fedora
and still incorporates the 'jpp' string.
One "de facto" procedure that is already used is that Jpackage
maintainers refer people to the Fedora guidelines for any case that is
not explicitly covered by some local rule. And the main reason for any
local rule is to make sure the other distros that are based on JPackage
are also OK with it (Mandriva, Suse...).
> Summary: It all comes down now to the remaining ".NNjpp" in the
release which indicates the upstream version that was last imported
(sync point you could call it).
In the case of the goals, I think the original ones in the week were not
correct in the first place. Lets try again:
" 1. Allow for upgrading between the Fedora and JPackage repositories
so that upgrade paths similar to the following works:
*
java-foo-1.0-1jpp_2fc [Fedora Release]
java-foo-1.0-2jpp [Updated JPackage Release]
java-foo-1.0-2jpp_3fc [Updated Fedora Release rebased
against the new JPackage]
3. A third expressed goal that conflicts with 1. is to upgrade
Fedora packages only with Fedora packages and jpackage packages with
jpackage packages."
> These two goals can be made into a single one, lets say: "flexible
upgrade paths".
> There is no conflict whatsoever -- we currently do all these:
a. Both Fedora and JPackage (what is stated in 1, i.e., newest release
globally) by leaving all repos enabled (both fedora-core and jpackage)
b. Fedora only by disabling the jpackage.repo
c. JPackage only by using --exclude=*jpp* --excluce=eclipse* when
upgrading from fedora-core.repo and enabling only the jpackage.repo for
the Java packages update (people play with --enablerepo for that management)
You're not understanding Goal #3. The goal is to have a mixture of
Fedora and JPackage packages installed on the system. Running "yum
update" will update to the latest version of the software from the
repository which the current version comes from. Your (b) and (c) are
for the case where you want to draw your java packages from either
Fedora or JPackage, not both.
You mean, if a package is from Fedora, you update it with only another
Fedora package and if it came from JPackage only with another JPackage
one? Yes, perhaps someone may want that (I never encountered that
situation, but I think it may be useful). That is a letter (d) though.
(a) has no restriction where a package comes from, the newest one is
what is wanted, no matter from where.
BTW, I am sure it can be scripted currently with "*jpp$" used to
identify what is JPackage. We must be careful with what we define for
pre-release tags for that to continue to work though.
Goal #3 could probably be realized with commandline switches if Fedora
did not include jpp in the name: yum update --disablerepo=jpackage
--exclude='*jpp*'; yum update --enablerepo=jpackage '*jpp*'
Yes, that would work. It is only my letter (d) case though.
"2. Allow users and packagers to tell what JPackage release the
java package was based against."
> This is very important to us (Java team). We can quickly identify if
the version we have already has a fix or not, among other things.
Can you give a developer use case? We tried to imagine a use case for
developers and what we came up with was looking in cvs for the package.
At that point, there is no rpm package so there is no release tag.
Giving us a few use cases will help to see what will and won't work.
I'd like the other Java team members to comment on this. I only have a
share of the packages to maintain. That is how I control my packages
but I'd rather other people give testmony.
Deepak? Vivek? Matthew? Permaine? ...
I will have to add a goal that was not stated in the initial Wiki, lets
call it 4:
"4. Allow the switching of Java stacks as a whole"
> This is currently accomplished by using regular expressions based on
the *jpp* and eclipse* patterns.
Again, a use case would be nice. I'm imagining switching between the
jpackage java stack and the Fedora java stack and don't see how *jpp*
would help here.
Unless you mean it makes it easy for you to select all the java packages
on the system and replace them with something else? That strikes me as
a misuse of the release tag -- 1) Not all java packages will have the
jpp name as not all java packages will come to us through jpackage. 2)
If we do this for java, why not for python, perl, php, etc?
Several things in this paragraph.
1) Except for the eclipse package, which removed the 'jpp' tag to
conform to the guidelines and is now maintained (the version we use)
locally, all Java packages have the 'jpp' string and all come from JPackage.
As the system of packaging Java we use is the JPackage system, we
contribute all our packages upstream and import them. As long as
someone is willing to maintain a package there is no restrictions. This
helps us with solving any issues with other Java libraries need by that
piece, Java-related packaging issues and so one on a community of Java
developers and packagers.
2) Our Java packages (Fedora has a subset, we actually deal with almost
300) have a very complex graph-like relationship among them, creating
what people have bee calling a "ecosystem" that is mostly independent,
just requiring a JVM to be present. I don't know about python, perl,
php etc., I can only comment on Java.
0) There is also the question of using the release tag for that
identification purpose. There are several considerations: (i) there
are other uses for that string, so as if it is there why not use it
(ii) Some other strings in the release identify things as well, for
instance, anything marked with %{dist} tells a package has a variant for
that distribution; that would also be a similar abuse (iii) some of the
proposals suggested even the Provides tag; that would be a serious
abuse, it is used by one of the most basic mechanisms of RPM; the
release tag markings like %{dist} or jpp are innocuous (iv) RPM and you
have no adequate mechanisms at the moment that are so handy, visible and
easy to use at the moment.
P.S.: We should try and suggest enhancements to the new RPM and to yum
(perhaps even contributing patches) to allow this kind of functionality
in the future without the need of using the release tag. For instance,
I'd like to see easy ways to retrieve, filter on, etc. on tags like
Group, both for RPM and Yum.
> This is useful for development so we can try sets of packages built
with a different compiler for instance, without having to change all
spec files to bump releases, and to revert the system to the original
state afterwards. We use this a lot.
> There are also users that want to switch the collection of Java
packages to use a set compiled with a proprietary JDK for instance
(maybe a different version of Java), or something provided by the
software vendor of some major component they bought.
You're going to have to post the commands you use to achieve these
things as I'm having problems understanding how you're doing this based
on the release tag.
I've already posted the yum ones. The above is just a piping of rpm -qa
thorugh grep and things like xargs. The regular expresiions given to
grep are the key here.
OK, now the Wiki page states:
"Which naming guideline is chosen depends in large part on whether these
are judged to be worthwhile and attainable goals."
> Which I think is correct.
Discussion of the goals
"Goal #1
If JPackage were truly an upstream then we wouldn't want users to
upgrade Fedora Packages with JPackage. We have bugfixes, local changes,
and other patches that we add to packages to make them work better on
Fedora. Having users upgrade between upstream and our packages is not a
goal with any other upstream. gbenson states this [WWW] here although he
phrases it in the specifics of Fedora/JPackage instead of the general
Fedora/Upstream."
> We don't do anything to make packages run better on Fedora. We do
pre-compilation, which does help in the startup time. But there is no
real harm in revert to a bytecode-only package to get an upstream fix
until it shows up in pre-compiled form in a Fedora update.
This may be true in the RH-Fedora java group but it won't be true in the
Fedora-community java space unless the mandate from the FTC is to get
packages approved in jpackage before getting them into Fedora. Not
everything happens upstream. Not every upstream agrees with the
downstream. Not every upstream has the same time frame for updates as
downstream. We will carry local patches and changes at some point.
IMO that should be a recommendation (no mandate -- too many mandates
already). As the system that allows the proper Java interoperation of
all these things is done upstream, people should try and get their
contributions there first.
This is the same as wanting a new Perl module and not going through that
Perl modules community first.
Remember, there may be (we hope there will be) a large Fedora Java
community in the future. But a Fedora+other distros comunnity will
always be larger than that (just set theory), and the larger the
community the better the software/package produced.
"There could be times when the jpackage naming is not compliant in the
name, version, or release fields. When this happens, we would not be
able to attain this goal. For instance:
cryptix-asn1-20011119-4jpp_2fc.1.1.src.rpm. The version field must be
changed before importing otherwise the final release of cryptix-asn1-1.0
will not upgrade the package."
> That was a wrongly versioned package from an older JPackage release.
The maintainer was informed and the problem was fixed. It is now
cryptix-3.2.0-9jpp. In general, we have had no problems in getting this
kind of things fixed. I think this comment can be removed.
No. cryptix-asn1 is an example. It doesn't matter if the specific
package has been fixed.
I disagree. That is not representative and was a unique mistake made by
an individual on a certain date. That would never be propagated into
Fedora as it would be detected during the import. The upstream
community would be notified and would have fixed it quickly (which was
exactly what happened).
The question's contained here are:
1) What are jpackage's naming guidelines so we can see how they mesh
with our names?
No different from Fedora, RHEL or any other package distribution.
Actually, people have been referred to the Fedora guidelines lately.
2) If there's a bug with the jpackage naming what will we do until it's
fixed?
The time to fix that is negligible, As the Java team is maintaining the
packages upstream, they have commit access so it is very easy to fix
these things.
3) Do we want to force our packagers and reviewers to learn two sets of
guidelines, one for jpackage, and one for Fedora?
There is no conflicting guidelines at the moment that we are aware of,
except for the release tag and the pre-release tags that is being worked
out. Maybe we will have something else depending on that Epoch thread,
but not at the moment.
As I said before, except for some Java-specific things that are not
covered in generic guidelines, people have been refered to the Fedora
guidelines ad so far this has not raised any complaints from the other
distros.
"Goal #3
This is not truly implementable via the package release tag.
Separating the native libraries into a subpackage might have some
bearing on this."
> Yes it is implementable, as shown above.
You're right. Remove the jpp from Fedora package release tags and we
should be able to do this :-) Seriously, the goal is to have "yum
update" just work, so even with jpp removed, we can't achieve this.
We were talking about two different goals. This one yes, but the (a) in
my list is not. I deam that more important than this one (my letter (d)
now) that I haven't see anyone use yet (although I recognize it may be
useful).
"Goal #2
Users shouldn't need this information. The Fedora Package may have more
fixes than the JPackage one. It may be synced with more upstream
bugfixes. It may contain snapshots of JPackage work that haven't hit the
repositories yet because JPackage is working on a new major release. If
the package works without bugs then the end user is happy."
> Putting a set of compatible packages together is a major endeavor.
The Java team, as well as the folks from Mandriva and Suse have been
basically polishing the sets that have been created upstream.
Ain't that the truth! We have to do this for the entire distro. That's
a bit besides the point.
It is not you who are having to deal with all these Java packages, so
please don't speak for us. We have already a heavy load of work and
can't handle the results of a fork.
> Also, we make the fixes/changes upstream first, then import. This
way we benefit from feedback from people (both packages and Java users)
who know deeply some of these Java software libraries.
This is where we have divergence. You claim that all fixes go upstream
first. I claim that 1) this won't continue to be the case (all fixes)
and 2) it may not be desirable. Example: There's a showstopper bug in
a java package just before F7 is spun. The jpackage maintainer doesn't
think it's a showstopper or would rather wait for upstream to fix it
before committing a patch. We aren't going to hold F7 for it so we have
to fix it locally. Example: jpackage upgrades to a beta version of a
component while we want to stay on the stable release or vice versa.
We do what we do for any software: we add a local change in urgent mode
and immediately contact upstream so the fix is incorporated in their
next release. If the change gets incorporated into another form that is
better than our stop gag fix we correct that in an errata.
We've been doing this for things like debuggers, DBMSs, Internet Servers
of all sorts for 10+ years.
> Again: updating paths are optional, and it is up to the user to
choose. The current naming only allows that to be done if so desired,
which would be otherwise not possible.
Not true. Jesse Keating proposed a naming scheme which would work
equally well.
Lets discuss the goals first. How can we discuss solutions we we havent
agreed on the goals?
We will examine all the proposed solutions based on the goals we agree upon.
"Developers can use this information. But they aren't going to be
staring at the package name most of the time. When they checkout from
cvs, they'll have a spec file, not an RPM so the filename is no help.
They have to look inside the spec file at which point it's just as easy
to put the information in the %description, a Provides, or a comment.
Using a Provides makes it queryable from the rpm command line as well."
> The way we do now is by "rpm -qa | grep jpp" and variations. Or
similar pattern usage in the repos. We don't need to look inside the
spec file to know that the base for that release was a specific upstream
release.
What are you looking for inside the package repos? What is your rpm -qa
|grep jpp used for? The comment above was from the projected usage case
of a developer needing to look at a package in the cvs repository to
decide what version a package was built against. In that case, you
would need to look inside the spec. Help us understand what you're
doing so we can see where the other proposals fail.
If we know what is the base point by looking at the release tag we don't
have to look into a file inside cvs just to see that. And it can be
done at large with diffs we do between lists of package sets.
> Using things like "Provides" for things that are not their original
goal would be abusing that feature, and the information for that is used
by depsolvers and such. We should stay away from that. In a lesser
degree, maintaining a release in the description is not convenient and
error prone, and does not address the other goals anyway.
There's all manner of odd things in Provides.
Provides: jpackage(foo) = 1.0-1jpp
Would fit right in :-)
So, you rather "abuse" (in you words) the Provides tag? That makes no
sense and it does not attain most of the goals above.
With changes to future RPM and Yum releases we could do things based on
other tags like Group and so one. It is just not available yet.
Regards,
Fernando
--
Fedora-packaging mailing list
Fedora-packaging@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/fedora-packaging
