Re: Fw: Spoofing user_agent

A standard HTTP Request headers is : User Agent (without the underscore). --Shreyas On Thu, Nov 25, 2010 at 2:36 PM, Ron Piggott <ron.piggott@actsministries.org > wrote: > > Is this what you are telling me to do: > > header('user_agent: RonBot (http://www.theverseoftheday.info)'); > > Ron > > The Verse of the Day > “Encouragement from God’s Word” > http://www.TheVerseOfTheDay.info > > From: ash@ashleysheridan.co.uk > Sent: Thursday, November 25, 2010 3:34 AM > To: Ron Piggott ; php-general@lists.php.net > Subject: Re: [PHP] Fw: Spoofing user_agent > > You need to set it in the header request you make. Putting it in the script > you're using as a spider with ini_set won't do anything because the Target > site doesn't know anything about it. > > Thanks, > Ash > http://www.ashleysheridan.co.uk > > ----- Reply message ----- > From: "Ron Piggott" <ron.piggott@actsministries.org> > Date: Thu, Nov 25, 2010 08:25 > Subject: [PHP] Fw: Spoofing user_agent > To: <php-general@lists.php.net> > > I have wrote a script to generate a sitemap of my web site. It crawls all > of the site web pages. (About 30,000) > > I need help to spoof the user_agent variable so the stats program running > in the background ( “AWSTATS” ) will treat the crawl as a bot, not browsing > usage. > > The sitemap generator is a cron job. I tried the syntax: > ini_set('user_agent', 'RonBot (http://www.theverseoftheday.info)/'/); > > This didn’t work. The browsing was attributed to the dedicated IP address. > > How do I get AWSTATS to access this, such as other entries under the > “Robots/Spiders visitors” heading: > Unknown robot (identified by 'bot*') > > I don’t mean any ill will by changing this setting. Thanks for the help. > > Ron > > The Verse of the Day > “Encouragement from God’s Word” > http://www.TheVerseOfTheDay.info > > -- Regards, Shreyas Agasthya