Redirect search robots onto a dedicated cluster machine

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

To: "users@xxxxxxxxxxxxxxxx" <users@xxxxxxxxxxxxxxxx>
Subject: Redirect search robots onto a dedicated cluster machine
From: Pete Jones <Peter.L.Jones@xxxxxxx>
Date: Tue, 2 Jul 2013 09:39:28 +0000
Accept-language: en-GB, en-US
Reply-to: users@xxxxxxxxxxxxxxxx
Thread-index: Ac53B7nxWFFej42kQmOrKQy7ZdrPqQ==
Thread-topic: Redirect search robots onto a dedicated cluster machine

Hello

We have a small cluster of machines serving a wiki website. Search engine robots are allowed in to index the website and these create a lot of traffic that can affect the service. I would like to redirect all http requests from these robots to one of the machines in the cluster.

Any ideas for how to do this ?

Many thanks in advance for any advice

Follow-Ups:
- Re: Redirect search robots onto a dedicated cluster machine
  - From: john gale

Prev by Date: RE: Modifying Location header with mod_headers
Next by Date: Re: Recommended way to setup file-uploading, in a Webdav based share
Previous by thread: Modifying Location header with mod_headers
Next by thread: Re: Redirect search robots onto a dedicated cluster machine
Index(es):
- Date
- Thread

[Index of Archives] [Open SSH Users] [Linux ACPI] [Linux Kernel] [Linux Laptop] [Kernel Newbies] [Security] [Netfilter] [Bugtraq] [Squid] [Yosemite News] [MIPS Linux] [ARM Linux] [Linux Security] [Linux RAID] [Samba] [Video 4 Linux] [Device Mapper]