Sam,
Thanks for posting your review. It caused me to have a look at the
document, because I've been considering writing something of a missive
on my own. I have to say that I enjoyed reading this draft up to about
Section 3, and then I came to some problems. I'll spell those out in a
separate message, but I agree with you on one key point: this document
does not does not provide actionable guidance, and actually avoids the
use of 2119 language. Clearly that was a design goal. But I think in
the end it has led to a good INFORMATIONAL document, or even the
beginnings of a good text book, and not necessarily a BCP.
Also, I agree with you that interoperability is not the first and
primary goal. Clearly, and I hope Dave and others will agree, the
primary goal is the ability to roll out new useful functions and
services in a way in which they can be managed in a scalable manner,
where one has understood the network impact (or total cost, if you will)
for that service. And Section 2 provides a really nice illustration of
that, with regard to what happened at Berkeley.
I have another problem with 3.2: I think the SMI and other
registry-based forms of structured data may be overrated in some cases.
Sometimes what is being managed is very simple data, through a central
service. One has to question who the consumer of the data is prior to
determining how one encodes that data, or even what data is truly
necessary. Consider the case of an application that is simply calling
home to determine if there are patches. How much structured data does
that service need? In this context, 3.2 is often irrelevant to the
application running on some host, but it may be very relevant to the
central system that is doing the updating.
What I would suggest is that some additional context is necessary
throughout the document to indicate really who we are talking to. Are
we limiting our role to router management? What happened to the toaster
with an SNMP stack? Would we use SNMP today for that toaster? Who
would manage it? Well, let's put it in a more serious context and swap
toaster for major appliance like air conditioner or refrigerator or PDU.
Ok, and again. Thanks Dave and others for writing the doc.
Eliot
On 6/3/09 9:22 PM, Sam Hartman wrote:
I was assigned this document as a secdir review.
I'm not sure I have any comments in that capacity.
However, I do have significant comments on the last call of this document.
I appreciate the work that has gone into this document. People have
worked hard to find examples, cases and even pithy
sayings/architectural principles from many areas of the IETF. The
document tries to be broad and to look at a lot of options. I think
that it would be reasonable to publish this document as the current
thinking of the ops WG or ops area. It may even be reasonable to use
something like this as guidelines for network/routing/sub-network
layer protocols to think about management and operations.
However this document is not suitable for publication as a BCP because:
* It does not reflect practices across significant areas of the IETF
* It does not provide clear, actionable guidelines
* It is not sufficiently clear to be understood outside the O&M area.
My background. I have never formally been involved in the O&M area.
However, I have studied SNMP as a participant and AD for the ISMS Wg.
I have studied syslog as a user, operator, and as the AD who sponsored
many of syslog's base documents. I have studied netconf as an AD who
reviewed the spec. I've worked as a small enterprise operator. I'm a
chair of a WG (LISP) that needs to consider operations and management
issues. I believe that I should be able to read and understand this
document; I believe that I'm well within its target audience.
I got as far as the bottom of page 18 before giving up.
Here are details on my concerns based on what part of the document
I've read. I'm happy to invest significant time to get other
reviewers to evaluate whether I have valid concerns; if I get others
to read the document and consider whether I have a point, then I've
done what I set out to do.
1) It does not reflect practices across significant areas of the IETF
This is a concern that the document does not have broad enough
consensus to be a BCP. I believe that significant areas of the IETF
do not view management interoperability as a goal--much less a guiding
principle of management. I've been involved in discussions in the
Kerberos working group where we explicitly discussed this and came to
the conclusion that management interoperability was not something
anyone in the room was going to work on. We did work on an
information model which covers aspects where people believed some degree of management interoperability would be desirable. It does not cover monitoring, faults, or the like--only provisioning of the database.
Similarly, I'm quite certain that most web server vendors, ATOM
implementors, etc do not want management interoperability. I
understand that ISP operators very much do want management
interoperability. I think that we have a significant conflict here
and I think that we cannot approve such a BCP without addressing that
conflict. One possible resolution would be to find an subsection of
the IETF that actually agrees with these guidelines and scoping the
BCP.
Similarly, it has been my experience that the desire to standardize
information model semantics is not universal across the IETF.
My recommendation for determining if I have a valid concern here is to
get one or two WG chairs from each area to read this document and
comment explicitly on whether the approach to management and
managability outlined in this document is consistent with practice in
their area and WG.
Far too much of the text was specific to management of network
elements such as routers, switches and the like. The apps and
security area use LDAP for a lot of things that I think would count as
management in this spec. I don't know how to separate control plane
and data plane issues on my web server. Enough of the text seemed
specific to network management instead of service/application
management that I would find applying this document in such a context
more frustrating than helpful. This specific sub-concern could be
entirely addressed by properly scoping the applicability of the
document.
2) It does not provide clear, actionable guidelines
I was looking at this document and trying to figure out what it would
mean for my WG if this document was approved. As best I can tell it
would simply add justifications for discusses that would be hard to
debate fairly. Do I have to write up an information model for my
protocol? If the ops AD says I do, what grounds do we use to
determine whether that's reasonable? The answer may be "yes and you
don't get to disagree," but I can't tell from the document. If it's
going to be a BCP, that needs to be clear to me as a WG chair.
For what it's worth, I don't think we have the necessary consensus to
require people to write up information models and would not be part of
such a consensus at this time. If this is a list of things I might
want to consider then why is it a BCP?
If we want to come up with a set of things that need to be documented
when appropriate, then I could support that. I think we need to
clearly distinguish the normative process requirements from the set of
things to consider in such a case. The set of things someone might
interpret as normative in this spec is far too broad.
I do agree that
some of the concerns I am raising in this section could be leveled
against BCPs we've approved in the past--even BCPs that I sponsored in
at least one case. I don't think that diminishes the concerns: we try
and learn from our mistakes.
3) It is not sufficiently clear to be understood outside the O&M area.
This document does an excellent job of clearing up a few o&m concepts:
the difference between information model, data model, modeling
language and protocol.
The document indicates that several distinctions are important, such
as the distinction between operations and management, the distinction
between configuration and other forms of management, etc. The
configuration vs non-configuration management distinction seems very
important because I believe there is a growing belief that the set of
management protocols you use depends on what you are managing. I hope
no one plans to use syslog to configure their routers--perhaps the
only worse choice would be IPFIX.
Unfortunately, while I agree that these distinctions are important, I
don't think the document succeeds in making them. I have reread
section 2 and the first part of section 3 a couple of times. I still
don't understand the difference between operations and management;
section two talks about management a lot and section 3 talks about
my operations model. I don't understand what an operations model is,
even after reading the text a couple of times.
I found section 3 very frustrating. I'd be reading along thinking I
was talking about management for one purpose, thinking about how that
interacted with protocols for that purpose, and then suddenly
something gets thrown in that completely fails to fit. For example,
I'd be thinking about how to manage configuration state and then
suddenly I'm being told to define the semantics of my counters
consistently. Both of those are fine things to discuss, but not in
such a way that a reader gets them mixed together. The mental context
swap is too jarring and I found completely lost me.
In conclusion, I'm sorry that I could not be more constructive. I
really a lot has gone into this document. I realize that the goals
are important. However I don't think we're particularly close to done
at least if the target is a BCP.
_______________________________________________
Ietf@xxxxxxxx
https://www.ietf.org/mailman/listinfo/ietf
_______________________________________________
Ietf@xxxxxxxx
https://www.ietf.org/mailman/listinfo/ietf