Andy, That is consistent with my memory. And all of those would qualify more as Applicability Statements, not actual conformance or compliance tests where there is reference code and/or scenario-generating code to test against. So, to repeat my point, Miles's question about "how we got away from writing compliance tests..." appears to be based on an incorrect premise because we never did write such tests. thanks, john --On Sunday, October 23, 2022 10:30 -0400 "Andrew G. Malis" <agmalis@xxxxxxxxx> wrote: > John, > > The closest we ever came were the various host and router > requirements RFCs (1122, 1123, 1812, etc.), plus a few others > here and there like RFC 7766. > > Cheers, > Andy > > > On Sat, Oct 22, 2022 at 6:55 PM John C Klensin > <john-ietf@xxxxxxx> wrote: > >> >> >> --On Sunday, October 23, 2022 08:18 +1100 Lloyd W >> <lloyd.wood=40yahoo.co.uk@xxxxxxxxxxxxxx> wrote: >> >> > >> >> On 22 Oct 2022, at 03:43, Miles Fidelman >> >> <mfidelman@xxxxxxxxxxxxxxxx> wrote: >> >> >> >> It does lead me to wonder about how we got away from >> >> writing compliance tests for everything. And, a follow-up >> >> thought that, perhaps, a compliance test might be a nice >> >> thing to add to RFCs (at least those that proceed to >> >> standards) - with IETF or ISOC or maybe ICANN maintaining >> >> a reference server to test against. >> > >> > Well, it's "rough consensus and running code" and not >> > "rigorous compatibility and RFC compliance". >> > >> > last I checked, standards required two separate >> > interoperable implementations - so which one would be >> > favoured and chosen for your reference server? >> >> Another reason is that there has been a great deal of >> experience showing that, if an SDO produces compliance tests, >> those tests, and not the written text, rapidly become the >> standard. If the test does not cover absolutely everything, >> that can be bad news. It gets particularly problematic for >> IETF specs because it is hard to express "SHOULD" or "SHOULD >> NOT". I'm also unsure about "got away from" -- as far as I >> know, the IETF, as the IETF, has never specified compliance >> or validation tests for its standards track documents. >> >> john >> >> >> >>
