On 03/05/2010 04:49 AM, Kevin Kofler wrote:
> Doug Ledford wrote:
>> So, I'm going to reiterate my policy suggestion:
>>
>> Make Fedora releases (all of them) stable in nature, not semi-rolling.
>> Make rawhide consumable as a semi-rolling release itself.
>
> And let me reiterate my objections, because you asked for it. :-)
>
>> Reasons:
>>
>> 1) Most importantly, it's a means of satisfying both groups of people in
>> the Fedora developer community and leaving none behind.
>> 2) It saves developers time and effort. This is actually the least
>> burdensome method of satisfying both groups of people inside Fedora.
>> The whole 2 update stream approach increases the load on maintainers,
>> while this approach reduces the load on maintainers as a maintainer gets
>> to consider a Fedora release "done" when it goes out the door with the
>> exception of fixing any security or important bugs in that release.
>
> Yet it is the only solution which really satisfies both groups of people.
You should always be more clear when writing emails such as this. The
"Yet it is" above is unclear. Are you referring to a stable rawhide, or
the two update stream model for satisfying both parties?
> Stabilizing Rawhide is pretty much a lost cause.
No, it's not. If we care about rawhide being stable, then it's up to us
to make it so. Since we don't currently care, developers do sloppy,
lazy things in rawhide and don't worry about it.
> What your proposal further
> down does is actually introducing an intermediate stream à la Debian
> testing, which comes with a lot of extra maintainer work as well.
It comes with less extra work than doing two update streams. Face it,
there is *no* solution to this problem that both solves the issue for
both parties involved and does not include at least *some* extra work
for you.
>> 3) It conserves resources (drastically so) on the fedora infrastructure
>> and mirrors and reduces bandwidth needs greatly.
>
> How so? You're still adding an extra repo.
The updates repos will be *drastically* smaller, enough so that the new
repo won't even come close to using up the space savings.
>> We have established, beyond any doubt whatsoever, that there are users
>> of Fedora that expect, demand, and need a stable update stream.
>> Ignoring those user's needs by saying "why can't we just stay as we are"
>> is tantamount to blatantly ignoring their cries for change/help,
>> trivializing their issues, and pushing them aside as unimportant.
>
> Uh no. It's just saying that there are dozens of distributions out there
> already doing exactly that, they should just pick one. Fedora has never
> worked that way, why should we do so now?
They are fedora users today, ignoring them is exceedingly rude. And
given the fact that more people have stood up requesting a more stable
update stream than those that have stood up for the semi-rolling update
stream, it is both very cavalier and risky on your part to throw around
the "we like it the way it is, go away" card. In the end, the one going
away may not be the one you want. It would be to everyone's benefit if
we didn't have either group go away.
> And I also dislike the usage of the term "stable" for this purpose, as it
> implies our current updates are "unstable", which to most users means
> "crashy".
Did you miss the KDE update thread yesterday? I stand by the name
stable, and I raise you an "it *was* crashy".
>> We have established, beyond any doubt whatsoever, that there are users
>> of Fedora that expect, demand, and use fedora *because* they get major
>> updates in the middle of a release. Ignoring these users needs by
>> saying "rawhide is ==> way" without first doing what is necessary to
>> make rawhide usable and consumable is tantamount to blatantly ignoring
>> the untenable nature of your suggestion, trivializing their wants and
>> issues, and pushing them aside as unimportant.
>
> Sure, but making Rawhide consumable is a lost cause (and is in fact not what
> your proposal is actually trying to do).
My proposal *is* trying to do that. Whether or not it is a lost cause
is a function of *us*. We make rawhide consumable, or we make it a
minefield. Either way, it's our actions at work. All we have to do is
care.
>> Technical implementation proposal:
>>
>> 1) Make rawhide consumable.
>> A) Create rawhide-unstable. Any time a known disruptive change is
>> being worked on, it should be built here by the developer. In
>> addition, add rpmdiff checks to all builds from devel into
>> dist-rawhide and if any of certain rpmdiff checks trip positive,
>> move the package from rawhide to rawhide-unstable. Additional
>> checks can be added as AutoQA gets into full swing, and so we can
>> add more ways to catch breakage and move things to rawhide-unstable
>> as needed.
>
> At this point you've created 2 rawhides, which already means 2 repositories
> etc.
So? Rawhide-unstable is just a temporary holding area. Unlike the
GA/updates repo combinations for releases, rawhide is not frozen and
doesn't contain a static set of packages. That means that
rawhide-unstable is not like an updates repo where it holds the updates
forever. Instead, packages move from rawhide-unstable to rawhide on a
regular basis, and when that happens, the rawhide-unstable repo shrinks
(preferably to 0, but if some of the packages aren't ready for the move
yet then they stay in rawhide-unstable). And given the amount of space
we'll save by not having huge numbers of updates in all the -updates
repos, the temporary space consumed by rawhide-unstable is totally
negligible and not worth even talking about. So, this objection is
meaningless.
>> B) Non disruptive changes go into rawhide directly, and on a regular
>> basis we run a compose on the rawhide tree to create install
>> disks/images for use. I suggest once a week we recreate the
>> images.
>
> (As Till Maas already pointed out: ) How do we verify these changes as non-
> disruptive without a testing repo (which would be a THIRD rawhide)?
You don't test locally before you build into the build system? That
would be one way. The other way is to actually know and understand the
code you are submitting. Many disruptive changes you simply *know* are
disruptive because you know and understand the code. This would be
determined on a maintainer by maintainer basis. Then you would also
have the AutoQA/rpmdiff checks on the packages post-build that would
attempt to catch disruptive changes that the maintainer missed.
But, really, this boils down to what I said earlier. If we care about
rawhide, then we make an effort not to break it, it will be better. If
we don't, then it will stay the same. There's no great technical hurdle
here, only a social one.
> Handling the 2 separate buildroots could also become fun, especially if we
> try to use rawhide-unstable as the testing repo (which isn't that great an
> idea because it won't allow easily testing the changes with the set of
> packages actually in rawhide-consumable, but which is the only choice in the
> absence of an additional rawhide-testing).
Sure it would. I'm not entirely sure why you think it wouldn't be easy
to pull packages from rawhide-unstable and run them on rawhide. As I
pointed out above, rawhide would regularly dump packages from
rawhide-unstable to rawhide when they are ready. That keeps rawhide and
rawhide-unstable so close that for the most part anything in
rawhide-unstable *should* work on rawhide with the obvious exception of
soname bump type issues. But, that's also part of the reason for having
rawhide-unstable, and in the case of soname bumps you simply have not
only the library that's bumping versions but also all the dependent
packages in rawhide-unstable so that the entire group can be updated on
top of rawhide and tested. This would work perfectly fine.
> We'd probably have to rebuild
> packages to move them into the consumable repo,
No. As above, most packages should be close enough to rawhide to move
safely, and soname bumps and the sort will all come as a group so when
you move the updated library, you move all the dependent apps at the
same time, no rebuilds are necessary.
But, to be perfectly clear, here's what I'm envisioning:
The devel branch builds into rawhide by default. The rawhide buildroot
inherits all packages from itself, so anytime a build goes directly into
rawhide it is immediately available in rawhide. The rawhide-unstable
buildroot inherits from rawhide and from itself, so anything in
rawhide-unstable overrides whatever is in rawhide. It also builds into
itself, so builds into rawhide-unstable are immediately visible in
rawhide-unstable. If you have a package you know will be disruptive,
you can opt to use make unstable-build instead of make build and it will
direct your build to rawhide-unstable. Let's assume for this exercise
that your change is a major update to python where all the dependent
packages will need rebuilt/touched up. You built your new python in
rawhide-unstable. You then notify devel@ that you have updated python
in rawhide-unstable and request all python module owners to rebuild
their packages in unstable against your new python. You could even file
bugs to this effect. You then wait for them to do their stuff. If they
are too slow, you could check things out in cvs and do a bump and
rebuild into unstable yourself. At some point in time, when the
majority (or all if you are that picky) of these packages are rebuilt,
you can put out a call for testing whereby people can do:
yum --enablerepo=rawhide-unstable update python
which will pull in the base python package and all the dependent items.
People can then test your code. You wait for feedback from people,
resolving items as you go, building new things into unstable as needed,
etc. Finally, when you are satisfied that this is really at a stable
point, you file the bodhi ticket for your package plus all the python
modules in unstable with you, and at the next regular
rawhide-unstable==>rawhide update they get moved over.
See, the workflow isn't all that different from now except that
disruptive changes are collected together, tested together, and moved
together when we have a reasonable amount of certainty that they won't
break things. It's only slightly more work than the current rawhide
method is today, but it makes all the difference in the world in terms
of rawhide being consumable.
> leading to extra work (and
> thus tempting the maintainer to just leave the stuff in rawhide-unstable,
No maintainer will *ever* leave stuff in rawhide-unstable. The reason
is that this is a place where disruptive changes go in order to be
settled before going to rawhide. If the maintainer does not fix up the
disruptive part of the change (eg. by getting dependent apps rebuilt in
the case of an soname bump), then the package will not get updated in
rawhide. So let me be clear on this, rawhide-unstable is not like a
testing repo, where if you don't like rawhide-unstable you can skip it
and go straight to rawhide even if your change is disruptive. If your
change is disruptive and either you voluntarily build your package in
rawhide-unstable (what you *should* do) or AutoQA/rpmdiff trip positive
on those tests we know for a fact are *always* distruptive (like an
soname bump that breaks dependencies from other packages) and your
package gets sent to rawhide-unstable, then the only way to get out of
unstable and into rawhide is to take care of the disruptive elements and
get your package plus whatever is necessary to clear up the disruption
moved as a group at the next checkpoint (or to back out the disruptive
changes and rebuild a new package that doesn't get sent to
rawhide-unstable, but you can't simply throw a disruptive change into
rawhide because you feel like skipping unstable).
> which will lead both to unhappy early-adopting users and to huge dumps of
> semi-broken packages on flag day,
No, if the breakage isn't fixed, it isn't included in the move on flag
day. The flag day is not simply a blind copying of all packages from
rawhide-unstable to rawhide, it would more likely be a regular bodhi
push where the people responsible for a package would have filed a bodhi
ticket listing all the packages necessary to resolve the disruption in a
bodhi ticket and that ticket would get pushed from rawhide-unstable to
rawhide. That adds a little extra work if you have a disruptive change
in that you now would have to file a bodhi ticket, but I see this as a
good thing because it forces the developer to do their homework, to
check that the issue is resolved, to account for all the packages that
need changed in resolving the issue, and to proactively notify rel-eng
that the change is actually ready. These are all good things even if
they mean an extra 10 minutes of work for the maintainer.
> we've seen exactly the latter happening
> again and again when the one Rawhide switched from Fn to Fn+1 in the pre-NFR
> days) and potentially invalidating part of the testing.
>> C) On a regular basis, we have a flag day to move items from
>> rawhide-unstable to rawhide. I originally said as-needed in my
>> first proposal, but on more reflection I would like to suggest
>> we make this a regular scheduled event on a monthly basis.
>
> This means we'd have 6 times as many flag days for disruptive changes than
> for a model based on non-conservative updates to stable releases. I agree
> with Till Maas that this is undesirable. Once again: what I, and I think
> most others in the "updates should be semi-rolling" camp, want is the new
> features that DON'T break anything, WITHOUT the breakage.
You yourself have proven that this isn't possible. Again I reference
yesterday's KDE thread.
Now, as I pointed out, there is the distinct possibility that by doing
this six times as often, each individual time will be much easier to
deal with, less prone to system breakage, etc. You may find, to your
surprise, that it is actually a better way of doing things. We don't
know, no one's ever tried it.
> An additional issue I see with the proposal as a whole is that I'm not
> convinced that, even if all the problems I pointed out above are solved,
> this will actually lead to something more consumable.
What I wrote above will absolutely be *more* consumable than what we
have today. Whether or not it would be consumable *enough* to use,
that's a different matter. But I guarantee the above steps would
improve things.
> The branched F13 right
> now is quite similar technically to your "consumable rawhide" and it even
> has the additional testing repo I hinted at, but it's still not something
> I'd want to run in production! For example, PackageKit 0.6 just became
> usable at all very recently. Now of course this may be due to the fact that
> there was nothing like rawhide-unstable before the branch event, but let's
> face it: we'll still always have stable releases to care about. At feature
> freeze (or on the last flag day before feature freeze), the disruptive
> changes which are targeted to be in the stable release WILL get merged, even
> if, when considering rawhide-consumable on its own, it'd be better to wait
> for a later flag day. And so we'd also have stuff like PackageKit 0.6 in
> rawhide-consumable. Unless of course you want to branch releases directly
> from rawhide-unstable and have rawhide-consumable be completely separate,
> but I don't think that particular idea is that great either.
>
> Rawhide is for development of future stable releases, so we import things
> like prereleases of the KDE version which will ship in that future release,
> which is essential so it gets testing by somebody, but which shouldn't be
> dumped on non-tester users, even early adopters. You're trying to give it a
> dual use, to also target early-adopting users, but that dual use is
> inherently flawed and an extra rawhide-unstable does not fix that, it just
> introduces extra work.
Most of these last two paragraphs seem to be directed at your vision of
my proposal which, as I pointed out many paragraphs above, is not the
same as my actual vision (aka, the whole part about the moving things
from rawhide-unstable to rawhide, which means they aren't really two
repos, just that one is merely a temporary holding ground before a push
moves stuff over to the other, very much unlike a GA/updates repo pair).
So, a few more items to help clear things up:
1) rawhide and rawhide-unstable would be considered one release, meaning
that if you build foo-1.2-1.%{?dist} in rawhide-unstable, you couldn't
build the same package in rawhide as it would be a tag collision.
2) #1 means that rawhide-unstable packages are only preferred if they
have a higher e-n-v-r than the rawhide package.
3) #1 and #2 mean that we may need to refine our cvs branches as they
relate to devel. In particular, let's assume you have package foo, and
you are preparing for a disruptive change from foo-1.2 to foo-2.0, so
you check in your changes in devel and build foo-2.0 into
rawhide-unstable. Then along comes a security issue in foo-1.2 but
foo-2.0 isn't ready to go yet, so you want to build just a security
fixed foo-1.2 and push it to rawhide immediately. We need a branch
policy that enables this as backing out the foo-2.0 changes entirely
just to build the security update would be a PITA. What ever we choose
this branch policy to be may effect the proposal in that instead of
having a make unstable-build we do two separate CVS branches or
something similar, but whether we use two branches or one branch and two
build targets is just technical details and is implementation dependent.
What I'm proposing here is really very simple and can be boiled down to
this: We change rawhide so that disruptive changes occur in a
coordinate, orchestrated manner instead of haphazardly. We can make it
happen so we can satisfy both groups of users in the Fedora community,
or we can choose to be lazy and keep doing things the way we do today,
in which case one group or the other is likely to simply have to go
elsewhere to get what they want, especially if we change policy so that
we are consistent in that either we mainly do or don't do updates in a
stable release.
--
Doug Ledford <dledford@xxxxxxxxxx>
GPG KeyID: CFBFF194
http://people.redhat.com/dledford
Infiniband specific RPMs available at
http://people.redhat.com/dledford/Infiniband
Attachment:
signature.asc
Description: OpenPGP digital signature
-- devel mailing list devel@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/devel
