On Saturday 28 May 2005 01:54, Les Mikesell wrote:
> On Fri, 2005-05-27 at 19:13, Lamar Owen wrote:
> > For the most part, the Red Hat crew is the best in the business. Or have
> > you never heard of Jakub Jelinek, Alan Cox, Rick van Riel, and many many
> > other of the top upstream developers that are employed in some capacity
> > by Red Hat?
> I think we've beaten these topics to death, but since it is kind of
> fun if you don't take it too seriously: Which of these guys knows how
> to make perl do character set conversions correctly better than the perl
> team?
Given time I believe Tom Lane (one of the PostgreSQL core developers and one
of the key libjpeg developers -- and a Red Hat employee) could do it without
question.
> Sigh... at this point it is "how many versions does the vendor support"?
> And the issue is that the perl version (among many other things) that
> does a certain operation correctly is only included with a kernel
> version that has features missing and broken.
And what's the solution? Wait until upstream gets it 'correct' for 1000
pieces of upstream? One of the packages is going to always be broken,
statistically speaking. You have to draw the line somewhere.
> I don't see how you can call setting up a WAN with many CIPE nodes, then
> finding it unavailable in the next release 'long term stability'.
You then stick with the five year cycle of the release you have installed.
You don't upgrade. In fact Red Hat specifically says not to 'upgrade' and to
carefully roll out version increases with scratch new installs. Then, if you
absolutely must have a feature only available in the latest release, you must
do a cost-benefit analysis to determine which features are most important for
you.
The five year supported cycle with no in-cycle version upgrades is the long
term stability I'm talking about; IT Directors (I am one, so I can speak like
one) want a stable five-year plan; and we do NOT want version upgrades that
can break things. Things like OpenSSL's versions, for instance, where the
ABI is broken between minor versions, even, are unacceptable for mission
critical servers (which IS the target audience for RHEL!)
The perl code change you mention could have broken many things; Red Hat or any
other Enterprise Linux vendor must freeze packages at some point to get any
reasonable amount of QA in; and you really don't want to mess up the pot in
a security release.
> > Microsoft-compatible PPP over L2TP over IPsec, which is so easy to set up
> > on the Windows client side it isn't even funny). That's why for-purpose
> > firewall/VPN appliance Linux dists (SmoothWall and Astaro, for instance)
> > are not using anything but IPsec. I have a SmoothWall box myself, and it
> > Just Works.
> Can you run it through NAT routers?
Yes. NAT traversal is fully supported.
> Whatever happened to the Caos distribuion? I had some hopes that they
> would combine a very stable kernel/libc combo with very up-to-date apps.
CentOS and cAos are no longer associated. cAos is at caosity.org and is still
doing their thing.
> > For instance, suppose you want PostgreSQL 8 instead of the supplied 7.4.
> So how is it better that when you do get Postgresql 8 in a distribution
> you will likely get a largely untested kernel along with it?
Just because the corner case that impacts you doesn't work does not mean the
kernel was largely untested; there were in fact many months of testing behind
the kernel that is in RHEL4/CentOS4. Your particular case was not tested;
did you bother to test the Fedora Core kernels leading up to it, knowing that
Fedora is the 'testbed' for RHEL? There's no such thing as a free lunch;
either you buy your lunch, you make your lunch, or you eat what someone gives
you. And beggars cannot be choosers: if the person who gave you your lunch
uses a seasoning you don't like and you complain, do you think the person is
going to make all their lunches your way next time?
> Or that
> you compile your own version and end up with potentially unique problems
> that every user has to solve individually?
When you compile your own you become responsible for what you have compiled.
This means you lose support from Red Hat, if you have an SLA for RHEL. This
means the developers of a free alternative don't have to support your
version. You are indeed on your own. That is the cost of open source,
unfortunately. You want support? Then you work within the framework you
have purchased or have decided to use. You don't like the framework? You
get a different one, and absorb the costs of the migration. Basic IT
business.
> I used to laugh at the people running windows servers because they would
> not even try to run more than one application per machine. But that was
> when I compiled a lot of stuff by hand. Now that I've gotten lazy (or
> smart...) and try to stick to binary distributions, I don't laugh any
> more because now my Linux boxes are in exactly the same shape. I can't
> count on being able to upgrade any single application without breaking
> another one.
That's right. It's not a Windows disease; it's a server disease, and the two
items on the balance are version stability and features. The two are often
mutually exclusive.
> Fedora breaks everything at once, so it isn't what I want. I want
> "something like" being able to install a Centos 3.x base, then
> selectively update certain apps to current-fedora level. A lot of this
> would work with just a source rpm rebuild - but what I'm really after
> is something that once done would still automatically pull updates
> when the selected packages were fixed.
Like Johnny said, then you probably want something like Gentoo or a BSD.
> I think character set conversion is something where the correctness can
> be determined objectively. And perl on Cento3 and Centos4 will do 2
> different things. I'll take the perl team's word for it being made
> correct in 5.8.3 and up.
So they do different things. There are many many things in CentOS4 that will
do different things from CentOS3; PostgreSQL, for instance, won't upgrade
between the two versions and you have to do a dump-initdb-restore between
versions. When the great Apache divide occurred (1.3 to 2.0) that broke
things; Sendmail version increases can break things; PHP version upgrades can
break things; how is the perl example any different? More to the point: was
it documented in the release notes? If not, when the first update came out
had it been noted? Was it noted during the development cycle (Fedora Core
being the development cycle)? You don't mean you tried to roll out an fully
unsupported upgrade (Red Hat DOES NOT SUPPORT and WILL NOT support upgrading
one RHEL version to another!) remotely using the first release of a major
version increment?
Well, if there is a critical feature you absolutely must have, then you really
really have a problem. I rolled out CentOS4 on a mission critical server:
but only after I had been using Fedora Core 2 and 3 for a long period of time
(FC2 and 3 are the development lines leading up to RHEL4, after all, and that
means the 2.6.x line had gotten right at a year of testing at my site prior
to rolling CentOS4 out!) on other non-critical but similar servers. It is
irresponsible to roll out releases that one has not tested to one's critical
servers, and in my shop could be grounds for dismissal.
> I have a feeling that the 2.6 kernel will become usable before that
> happens this time around, but maybe by the next time RH rushes out
> a largely broken release there will be viable competition in Ubuntu
> and other distributions that will keep a working kernel under a
> fast release schedule for the apps.
Quite frankly, if RHEL4 were 'largely broken' it would not have been released.
It is not a perfect release; none are. But it is a pretty good release and
meets my needs very well: to the point that CentOS 4 is becoming the standard
desktop and server Linux for PARI (as well as for me personally). Just wish
a SPARC version existed for my cluster of 20 Ultra 30's.
> I don't see how the CIPE author has any reason to care one way or
> another whether any particular distribution includes his package or
> not. Why should he?
Because his users might care?
> Well, if I had previously included CIPE, promoted it's use, and wanted
> anyone to think the distribution had long term stability...
RHEL3 has long term stability: within RHEL3's line (and RHEL3 is STILL BEING
SOLD and SUPPORTED for five years from initial release) there is long term
stability. Red Hat doesn't support upgrades from RHEL3 to RHEL4, and fully
documents the lack of CIPE in the release notes, which everyone should read
before deploying an OS anyway. This is the way Big IT works, likes to work,
and needs to work. You want the stability of a Big IT OS, then you must
either take what's distributed or put the responsibility on your own
shoulders. Or find a distribution that better meets you needs (like Johnny
said, Gentoo might be your best shot, or, if you like Ubuntu go that way:
it's Open Source and you're free to choose (that is, after all, what I like
most about Open Source, is that I am free to choose what meets my needs, and
can really do a cost-benefit study on various options)).
> Well, now we know how much value they place on maintaining
> interoperability with their own older products. Don't forget that
> when planning for the future.
This is very true. And is simply one of the costs of doing business; they
have to target their customers, and for the most part they are doing a great
job of it. Yes, they make missteps (caching-nameserver anyone?) but the
positives are far greater than the negatives, IMHO and for my particular
purpose. YMMV. CIPE wasn't high enough on their customers' lists to
override their other concerns; they made the executive decision to drop CIPE
since it was going to be a pain to get it working, and they wanted to go
another direction anyway.
> Please explain how the CIPE author has any vested interest in this.
> I would have expected Red Hat to try to provide their existing
> users a way to upgrade one site at a time and remain compatible.
> Now I don't expect much at all.
Red Hat makes no secret about their complete non-support of major version RHEL
upgrades; it's plastered heavily all over their release notes, and, in fact,
you must supply a kernel command line parameter to get any upgrade
functionality in RHEL. This should tell you something; I know it tells me
'upgrades aren't supported: if they work, you got lucky.'
Further, even with Fedora upgrades aren't really supported (but, then again,
the whole OS isn't supported).
This is the route Red Hat has chosen to go; for most of their paying customers
it is the correct behavior.
> > Linus != Red Hat. This is a Linus issue; not a Red Hat issue. Thus the
> > subject line. You can't blame Red Hat for something Linus caused.
> Red Hat made the decisions about if and when to ship it.
And they were the last Enterprise Linux vendor to ship it. Their customers
wanted the featureset 2.6 provided, and CIPE wasn't high on their list.
> CIPE doesn't have to play any game, but a version for 2.6 is available
> and has been for some time. And before you say things had to be in
> Fedora before shipping in RHEL, look at the Mysql version included in
> RHEL4.
MySQL is more important than CIPE to the majority of Red Hat's customers, and
MySQL 4 was considered a critical feature by Red Hat. I myself don't use
MySQL (I use PostgreSQL, which to me is much better in every way). Red Hat
was getting beat up for the whole MySQL problem (which was a MySQL AB caused
problem), and they really had to correct it. That might very well bite them
in the future.
> I think I asked before what things are measurably better in 2.6 and
> didn't get any answers. Are there any?
Sure. LVM. IPtables. ATM support. SMP. NUMA. HyperThreading (which really
isn't enterprise-grade yet, unfortunately). Sizeably better performance in
the core LAMP role. Seriously better performance (nearly twice) in typical
file server roles. Red Hat is much much closer to upstream kernels in the
2.6 line rather than the very heavily patched and nearly 2.6 RHEL3 2.4
kernels. Filesystem size not limited to 2TB. Support for more CPU's in SMP.
Support for much larger physical memory. More devices and processes
available for larger servers. Better interactive response for desktops.
See http://www.2cpu.com/articles/98_1.html and http://librenix.com/?inode=3972
for more. Yes, 2.6 is worth the pain for the gain. Both of these articles
were written nearly a year before the release of RHEL4; sounds to me like it
wasn't an 'untested' kernel. Also see
http://www.infoworld.com/infoworld/article/04/01/30/05FElinux_1.html
But I do somewhat sympathize with your position; after all, I still have a
mission critical server running Red Hat Linux 5.2. Yes, 5.2. Long ago in
the mists of time support stopped; but I have a mission critical binary-only
application (that will remain nameless) that won't run on anything higher.
We're attempting to find a replacement, but none exist that is backwards
compatible to some of our clients; we have the later version for newer
clients, but keep the old server running for the clients that won't work with
the newer version.
The reason? The app was actually written for Red Hat 4 (I've come full
circle; I started out with Red Hat Linux 4 and am now, eight years later,
running a clone of a Red Hat 4 distribution!) and the compat libs needed
won't work on Red Hat 6 or higher. There was some work to get it running on
Red Hat 6.2, but it wasn't stable. The 5.2 server runs like clockwork; but I
have to be extremely careful to packport highly critical fixes (2.0.40
kernel, for instance). And I can't run any Internet-visible apps on it; too
old; it's so old it doesn't even have SSH! So I had to pull in a second
server to run the Internet-visible stuff. That was the cost. This app's
support of Red Hat 4 was, by the way, one of the reasons I bought Red Hat
Linux 4.1 in the first place.
Also, I am in a similar position with newer versions of software; in my case,
there is a repository catering to my needs: KDE-Redhat. I need the features
of KDE 3.4 (Kstars in particular) and Rex does a great job keeping up. But I
wouldn't deign to ask the core CentOS team to maintain a KDE 3.4 release and
the over 500MB of needed files (it touches so many things you must download a
vast array of packages to just get KDE 3.4). If Rex (Dieter) weren't doing
it, I would have the difficult choice of whether to do all that work myself
or do without the features. In that particular case I would probably switch
to using Xephem instead of Kstars and maintaining it myself, but even then
I'd have to look at the decision very carefully.
I also have applications that would like to have Python 2.4 available; but
CentOS 4 comes with Python 2.3. When one of the core Python apps I use
(either GNUradio or Plone/Zope) begins requiring Python 2.4 then I will have
another choice to make: are the features of the newer versions of GNUradio
and Plone worth the effort of maintaining my own Python 2.4 installation?
I'm already maintaining my own fftw3 distribution (GNUradio requires the
single-precision build, which is not the default), so I can do this (I
maintained the PostgreSQL RPMs for five years (not anymore; people were
demanding too much backwards compatibility for me to realistically continue
volunteering my valuable time!) so I know what I'm doing). But the question
distills to whether the features are worth it or not. And I do not expect
the CentOS team or Red Hat to do any of this for me; if an upgrade breaks
something I have done then I'm on my own.
I have also done this before with Tcl; I run an application server framework
called OpenACS at a couple of sites, and it requires Tcl 8.4.6+ built for
multithreaded operation; this is not the default version nor is it the
default configure switch.
And I don't use any of the RPMs for Zope; Zope is way too customized at my
site for that.
--
Lamar Owen
Director of Information Technology
Pisgah Astronomical Research Institute
1 PARI Drive
Rosman, NC 28772
(828)862-5554
www.pari.edu
