Two points here. Stephen Farrell is spot on in having some skepticism about mission statements. That said this entire thread points to a much larger and potentially fatal contagion that often infects organizations. Its called “mission creep”. Too often an organization wants to be all things to all people and ends up being nothing to anyone.
Testing/interoperability/conformance/compliance are a quicksand of issues. Vendors generally hate them since they typically involve large sums of money especially if the compliance testing involves branding rights (aka logos).
My question is what is the problem/failure with protocol X that having ISOC intervene with testing/interoperability/ conformance or compliance programs would solve? Typically, members of a specific protocol community have self-organized to look at these issues and generally have been pretty good at it. Case in point the SIP Forum and the SIP protocol. We’ve been around since 2000 (yep 17 years !). We’ve always sponsored and partially funded the SIP interoperability testing programs, SIPit run for many many years by Robert Sparks. We have always developed the implementation profiles the industry relies on for compliance, SIPConnect. We are now working with other SDO and national regulators on the STIR/SHAKEN framework. It’s a system that seems to work well. Sure ISOC could have been helpful but it has virtually no experience dealing with national regulators outside the context of the IANA transition.
I’m not saying ISOC shouldn’t be involved in implementation and compliance issues. It could be very useful in kickstarting efforts by specific protocol communities in self organizing if there is sufficient consensus to do so, but acting as an overall umbrella for such efforts IMHO is not the best use of ISOC resources.
Gonzallo is perfectly right in asking the community for input on what future directions ISOC should take and this is a useful thread but I urge caution.
—
Richard Shockey
Shockey Consulting LLC
Chairman of the Board SIP Forum
www.shockey.us
www.sipforum.org
richard<at>shockey.us
Skype-Linkedin-Facebook –Twitter rshockey101
PSTN +1 703-593-2683
On 10/28/17, 12:45 PM, "ietf on behalf of Russ Housley" <ietf-bounces@xxxxxxxx on behalf of housley@xxxxxxxxxxxx> wrote:
> On Oct 28, 2017, at 9:51 AM, Sam Hartman <hartmans-ietf@xxxxxxx> wrote:
>
>>>>>> "Russ" == Russ Housley <housley@xxxxxxxxxxxx> writes:
>
>>> Some of us were very badly burned, in one way or another, by
>>> formal conformance tests of OSI implementations the best part of
>>> 30 years ago. So while I fully support Michael on "facilitating
>>> conformance testing" and would even insert the word "rigorous", I
>>> would be very cautious about "formal methods" (except for things
>>> like MIB modules and YANG, where clearly a formal check is
>>> required).
>>>
>>> Interoperability remains, IMHO, much more important than formal
>>> correctness. Implementations can be formally correct but faulty
>>> in practice.
>>>
>>> In any case, I don't think that distinction is relevant to the
>>> ISOC mission, which needs to retain some level of abstraction.
>
> Russ> Care must be taken that ISOC does not create an unfortunate
> Russ> feedback loop by sponsoring standards development and
> Russ> conformance testing of implementations. Testing should
> Russ> provide feedback into the standards process, but testing
> Russ> should not drive the standards process.
>
> Russ, I suspect I probably agree with you. However, I spent a couple of
> minutes trying to understand what you wrote and realized that I was
> having a hard time. Could you give a couple of example of giving
> feedback into the standards process (what you'd like to see) and testing
> driving the standards process so I can see the difference?
I am aware of a situation in another standards organization where the organization that wrote the standards also did the testing. The testing group wanted to have some aspects of the standards changed (make it easier to conform) so that the testers could generate more revenue. The testers we being measured on the revenue, not making the ecosystem better.
Russ