[Reduced Cc: and change Subject:] On 26/09/17 13:11, Eric Wong wrote: > There's no blocks on public-inbox.org and I'm completely against > any sort of blocking/throttling. Maybe there's too many pages > to index? Or the Message-IDs in URLs are too ugly/scary? Not > sure what to do about that... > > Anyways, I just put up a robots.txt with Crawl-Delay: 1, since I > seem to recall crawlers use a more conservative delay by default: Not sure what made the difference: Google can now easily find this thread on both https://public-inbox.org/git and on spinics too. Same change observed for another, unrelated thread on this list. Nevermind. "Unsearchable are his judgments, and his ways past finding out"
