On Tue, 24 Nov 2015 10:46:55 -0800, Adam Williamson wrote:
> We have never really gone to any lengths to test or support N-1 upgrades
> with 3rd party repositories or non-repo software either. That's a
> different question.
>From a user's perspective, the value of system-upgrade depends on its
ability to move his system "as configured" to a new release. All of his
personalization, including any non-fedora software, remains in place.
You are certainly right to except non-Fedora software from QA tests. The
problem I perceive is the vast number of possible combinations of
packages from just the Fedora repositories. There may be no need to test
exhaustively, but any heuristic is likely to miss problems from time to
time. It is likely only enough cases can be tested to distinguish "some
system-upgrades work" from "system-upgrade is often impossible."
This is valuable to know, and I have no quibble with a QA criterion that
says "system-upgrade is never possible" will block a release. I suspect
there is no precise, useful definition that will tell a user whether his
particular system will succeed with system-upgrade. All he can do is try
and hope for the best. There might be documentation that says "these
circumstances are likely to prevent successful system-upgrade" - the
"common bugs" approach.
With any general claim "You can use system-upgrade to advance to the next
(or next+1) release" there will be triage issues (how many users will
experience problem 1; how many will encounter problem 2?) How many
susceptible users must we estimate in order to block a release? I fear
software developers will rightly claim they are busy with new features or
current bugs and they cannot spend time to investigate compatibility
problems from system-upgrade that simply do not appear when their sofware
is newly installed.
I do not want a Fedora user to understand "You can use system-upgrade to
migrate to a newer release" has the same confidence as "The new Fedora
meets all release criteria."
> Why does the situation inevitably 'get worse' for N-2 upgrade?
There are more possible initial conditions. Even a very sparse test plan
requires more effort. Analysis of "How important is this case?" may be
more difficult, and there are more cases. Instead of problems from six
months of software evolution, there will be problems from 12 months, and
the increase in number of problems over time is likely greater than
linear.
I certainly do not want to abandon system-upgrade. I want users to
understand we cannot test it well, probably cannot fix many failures,
and, when it fails, we reccommend they report the problem then undertake
a new installation of Fedora, configure it, and reload applications to
meet their requirements.
"System-upgrade must work before release" is too vague ("work" is too
difficult to define and test), too severe (important new function could
be delayed for reasons no one is willing to fix), and will tell users who
experience problems that Fedora is lower in quality than they hoped.
If users are told up front that system-upgrade is useful but cannot be
tested or guaranteed for all cases, their expectations will be more
realistic, therefore satisfaction with Fedora will be greater.
"dnf system-upgrade" is rather new. Maybe the Fedora organization should
observe user experience for another release or two before it decides this
is a critical, and therefore release-blocking, part of Fedora. I
understand enthusiasm for the new, offline upgrade mechanism. More
history will yield a better understanding of what problems can occur, and
what costs must be borne to fix them.
--
test mailing list
test@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe:
http://lists.fedoraproject.org/admin/lists/test@xxxxxxxxxxxxxxxxxxxxxxx
