Saturday, May 27, 2006

Scrapers Impacting Conversion Rates?

Was just reading on Threadwatch about the Shop.org report on the decline in conversion rates for online stores and suddenly had an epihany that scrapers may be involved in this equation.

Let's assume that these conversion rate facts and figures include many of the non-human stealth crawlers that I'm blocking on a daily basis. There's no way your average online retailer is probably aware of this situation and you know they're being scraped just like the rest of us, maybe even scraped MORE than the rest of us, who knows.

Using one of my websites as an example, it averages 13,500 visitors a day and 50-200 stealth crawlers are being blocked which accounts for .5% - 1.5% of my daily traffic, which would definitely impact the conversion rate for any store with similar traffic.

Perhaps scrapers on a very large website getting a million visitors a day wouldn't have much impact unless the site attracts a lot more scrapers than my site. However, a smaller online retailer with similar traffic to the site I'm protecting would obviously notice a difference in their conversion rate, a HUGE difference, just by adjusting their stats to include pages downloaded by stealth crawlers.

Just another example of how scraping and stealth crawling is BAD FOR THE WEB and needs to be stopped.

No comments: