A standard HTTP Request headers is : User Agent (without the underscore).
--Shreyas
On Thu, Nov 25, 2010 at 2:36 PM, Ron Piggott <ron.piggott@xxxxxxxxxxxxxxxxxx
> wrote:
>
> Is this what you are telling me to do:
>
> header('user_agent: RonBot (http://www.theverseoftheday.info)');
>
> Ron
>
> The Verse of the Day
> “Encouragement from God’s Word”
> http://www.TheVerseOfTheDay.info
>
> From: ash@xxxxxxxxxxxxxxxxxxxx
> Sent: Thursday, November 25, 2010 3:34 AM
> To: Ron Piggott ; php-general@xxxxxxxxxxxxx
> Subject: Re: Fw: Spoofing user_agent
>
> You need to set it in the header request you make. Putting it in the script
> you're using as a spider with ini_set won't do anything because the Target
> site doesn't know anything about it.
>
> Thanks,
> Ash
> http://www.ashleysheridan.co.uk
>
> ----- Reply message -----
> From: "Ron Piggott" <ron.piggott@xxxxxxxxxxxxxxxxxx>
> Date: Thu, Nov 25, 2010 08:25
> Subject: Fw: Spoofing user_agent
> To: <php-general@xxxxxxxxxxxxx>
>
> I have wrote a script to generate a sitemap of my web site. It crawls all
> of the site web pages. (About 30,000)
>
> I need help to spoof the user_agent variable so the stats program running
> in the background ( “AWSTATS” ) will treat the crawl as a bot, not browsing
> usage.
>
> The sitemap generator is a cron job. I tried the syntax:
> ini_set('user_agent', 'RonBot (http://www.theverseoftheday.info)/'/);
>
> This didn’t work. The browsing was attributed to the dedicated IP address.
>
> How do I get AWSTATS to access this, such as other entries under the
> “Robots/Spiders visitors” heading:
> Unknown robot (identified by 'bot*')
>
> I don’t mean any ill will by changing this setting. Thanks for the help.
>
> Ron
>
> The Verse of the Day
> “Encouragement from God’s Word”
> http://www.TheVerseOfTheDay.info
>
>
--
Regards,
Shreyas Agasthya
