Thanks to the authors for doing this work; having it documented in an IETF-stream RFC would indeed be a good thing. I'd be *much* more comfortable doing so if we had public indication from other consumers of this format (especially: competing search engines) that the definition is correct and interoperable with their implementations. Cheers, > On 1 Mar 2022, at 4:39 am, The IESG <iesg-secretary@xxxxxxxx> wrote: > > > The IESG has received a request from an individual submitter to consider the > following document: - 'Robots Exclusion Protocol' > <draft-koster-rep-06.txt> as Informational RFC > > The IESG plans to make a decision in the next few weeks, and solicits final > comments on this action. Please send substantive comments to the > last-call@xxxxxxxx mailing lists by 2022-04-07. Exceptionally, comments may > be sent to iesg@xxxxxxxx instead. In either case, please retain the beginning > of the Subject line to allow automated sorting. > > Abstract > > > This document specifies and extends the "Robots Exclusion Protocol" > method originally defined by Martijn Koster in 1996 for service > owners to control how content served by their services may be > accessed, if at all, by automatic clients known as crawlers. > > > > > The file can be obtained via > https://datatracker.ietf.org/doc/draft-koster-rep/ > > > > No IPR declarations have been submitted directly on this I-D. > > > > > > _______________________________________________ > IETF-Announce mailing list > IETF-Announce@xxxxxxxx > https://www.ietf.org/mailman/listinfo/ietf-announce -- Mark Nottingham https://www.mnot.net/ -- last-call mailing list last-call@xxxxxxxx https://www.ietf.org/mailman/listinfo/last-call
