(for those who don't like long notes, roughly the last half of this one is a pair of extended footnotes that can safely be skipped.) --On Tuesday, January 16, 2018 09:39 +0900 Randy Bush <randy@xxxxxxx> wrote: >>> So, can we go back to the design goals and mission given to >>> the folks who did this, get it posted >> >> It _was_ posted. And reviewed, and updated. The SOW is at >> >> https://iaoc.ietf.org/documents/IETF-Website-SOW-02.pdf > so, clearly that process could have worked a lot better than > it seems to have. what's to learn to reduce likelihood of a > repeat? FWIW, I reviewed that SOW carefully after getting Andrew's note and now remember it and think I even made some comments on it or its predecessors. It is clear to me that what we now have is not consistent with the way I interpreted the SOW when I first looked at it [1]. I think there are several actionable lessons implicit in that: (1) We need to be much more clear about goals and objectives. Many of the things listed as "objectives" in the SOW are means to implicit ends. "Implicit" may not work for us and we may need to rethink how we structure SOWs. That is easier said than done; if it implies skill sets we don't have or don't have mechanisms to apply, it has possibly-profound implications for the IASA 2.0 effort. It also implies that, unless the community is willing to delegate complete authority to committees that do not operate in public and transparent ways, actual contractual terms, not just the SOWs on which the contracts are presumably based, should be available in the interest of review and transparency [2]. (2) Especially if a contractor is not already deeply embedded in the IETF community and culture (and probably if they are), the language of an SOW or contract needs to be much more clear about definitions and, if there might be tradeoffs to be made, priorities. As just one example, it is now clear to me (and should have been clear earlier), that, if I say "modernize ... the website's ... design" to a commercial web site designer, I am likely to end up with something that looks a lot like a commercial marketing or user-as-product site, something we probably did not intend. (3) No matter how good a job one does of definitions, sometimes one needs to build models or prototypes in order to be sure all parties are talking about and expecting the same thing. That isn't specific to web sites but is probably a general engineering design principle. Stewart's suggestion about paste-ups and story boards are directly relevant here, but sometimes working models are needed for people to say "whoops, you didn't understand what we wanted, time for a reset". Probably that means that contracts that involve design elements should be written to specify benchmark points at which mock-ups or prototypes are presented, with possibility of changes or course or even cancellation if those monitoring the contract conclude that the contractor doesn't get it. That is my answer to "reduce likelihood of a repeat?". Others can probably add other things. As to what we should do not (and to partially review what I said earlier): (4) Treat what is up now, retroactively, as a prototype, being prepared to write it off as a learning experience (expensive and too bad, but these things happen). Go back and review the goals and objectives in the light of that prototype and decide how it compares to what we really wanted and adjust accordingly. As part of that effort, cut back on the nit-picking or move it elsewhere. Then decide whether we can best get to where we want to end up by adjusting the current implementation, going back and adjusting the prior one, or starting over. Then figure out how to execute on whatever decision we reach, preferably without another four-year process. >From experience and observations of other efforts, actually doing (4) will be hard for this community. If nothing else, complaining is easier that careful analysis and specifications and the design of the IETF web site is probably directly part of the day job of _very_ few of us. However, if we can't do it (or the IAOC can't figure out how to take the steps required), we probably have the website we deserve whether it is the one we wanted or not. And that may be another lesson for the future. best, john -------------- These notes can safely be skipped by people who don't like long notes or details. [1] Section 2 of the SOW that Andrew cited says the following (quoted with my comments interspersed): > 2. Project Goals > The IETF website redesign will: > a. Improve ease of navigation, If navigation by people seeking to do work is included, the current design is a fail. I note that it does not say "except that it is ok to make navigation harder for the more frequent users of the web site, i.e., the active participate in the IETF". > b. Modernize and improve the simplicity of the website's > visual design, See above about "modern". > c. Make the website work better for all classes > of devices, including smart phones and tablets, > d. Ensure the website works well for visitors using > low-bandwidth and highlatency connections, This is not consistent with large, high-resolution, images, at least unless the users in those constituencies are constrained to be very patient. > e. Adopt responsive design Whatever that means. If it means (or includes) "respond to IETF community requirements, I suggest these threads indicate that site design is not doing well. > f. Improve accessibility, and "Improve" is a little vague, but I think several comments on the list have suggested that this requirement has not been well-satisfied either. > g. Improve the content maintenance processes. This seems still uncertain to me (only partially because of "improve"), but the large number of 404s from links off the deployed home page does not suggest that the sanity checks that are normally part of good maintenance processes were incorporated into the implementation. [2] Yes, I understand that contract negotiation on the IETF list would be an even worse choice than technical or graphic design on the IETF list. I am not suggesting that. I am suggesting that this particular result may be an example that suggests that we have not gotten the balance completely right, at least unless the IAOC/ IAD can assure us that there are no substantive provisions in the contract that are not in the SOW, noting that any provisions for review of intermediate or prototype products, or for determining whether a delivered product in compliant with the contract specifications, are, relative to this SOW, substantive.