On Fri, 19 Apr 2002 16:00:32 EDT, Liao Wei <lwei@cpmail.cyberpathinc.com> said: > Is there any this kind of document for RFCs. If an internet device supplier > claim their > implementaion conform to RFCXXXX. What should subscriber do to verify the > conformance. First off, decide whether "RFC conformance" or "interoperability" is what you're actually looking for. Second, think about whether a conformance test will actually find the broken corner cases that cause problems in real life. Third, ask yourself which you'd rather have - a piece of paper saying it follows the RFC, or a piece of paper saying that they'll fix problems if you find that it doesn't work. As part of a project I'm involved in, I recently did a bit of basic design work for something that would be fairly close to conformance testing (you old-timers that were around for the UL discussion a while ago remember that ;) and convinced myself that although for most protocols it was quite easy to verify that a given implementation follows the protocol when everything works well, that really doesn't tell you much - there are very few vendors who can get away with shipping total manure-ware that's totally standard non-compliant in the trivial cases. The problem is that it's *very* *hard* to come up with good testing for the error cases - which is where all the real-world uglyness comes from. I started thinking about doing it for SMTP (which I think I know something about), and realized that I didn't know how to write tests for the sort of things that I've seen "in the wild". For instance: I've seen hosts that you can communicate with just fine when SMTP pipelining is off, but which wedge up when you turn it on. Reason? Some router in between is tossing ICMP Frag Needed packets with an RFC1918 address, which our border router then tosses, breaking Path MTU Discovery - and pipelining allows enough RCPT TO: in one packet to tickle the problem.. (Figuring out why this didn't bite during the DATA phase is left as an exercise for the reader ;) The worst part is that both ends did exactly what they were supposed to in the face of what looked like network outages - abort, requeue, and retry. I won't get into the theory-versus-practice issues of %-hacks, !-paths, @host: source routes, and what happens when mixing them other than to note that I once saw the inventor of %-hacking describe it as "maybe the evil that men do does live on..." ;) Bottom line - ask the vendor for sites that have installed it before, and see if they've gotten it to work with other vendor's stuff. That's the way we've always done conformance testing. If the vendor's stuff doesn't play nice, ask the vendor to fix it. If they won't, and they're a big enough vendor that they care about their public image, there's any number of journalists of varying ilk all looking for "Vendor XYZ problems with..." stories. ;) -- Valdis Kletnieks Computer Systems Senior Engineer Virginia Tech
Attachment:
pgp00061.pgp
Description: PGP signature