Monday, March 26, 2007

WorldWebWide Scrapes LookDumb

Today my content tracking bugs led me to something in the worldwebwide.net which originated from LookSmart.

The IP address where the data was originally crawled from:

60.88.242.64 -> sv-crawlfw4.looksmart.com
This is nothing new as LookSmart seems to be a scraping target as I've already reported same thing happening with GoodBidWords.com containing scraped LookSmart listings.


Livebot vs. Googlebot - Microsoft trying to catch up?

Was looking over my log stats today for my site (not this blog) it was surprising to see Microsoft's Livebot crawling very aggressively, possibly taking more pages and returning faster then Googlebot this month. I'll give Livebot this much, the volume of crawling is pretty impressive compared to what it used to be so it looks like Microsoft is in this horse race to win and not just play 3rd fiddle.

Then of course we have our pals over at Ask slowly poking around my site. If Yahoo's Slurp is taking a nap this month, Ask must be in a coma or something. Slow as a snail and information seems to take forever to show up on their site. Maybe they cater to the dial-up crowd.

Then good old Gigabot seems to be crawling from a new bank of IPs and I don't even care.

Anyway, it will be interesting to see how quickly some of my new site changes show up in Live because crawling that fast and not updating as quickly would be silly!

Sunday, March 25, 2007

Site Upgrade Finally Started

Finally got around to upgrading to the new Blogger layout today and even managed to organize some of the posts with labels just to see how that works. The archive section is much easier to navigate and I think people will find this a whole bunch easier to use once I'm finished with upgrading.

Now to find a nice stretch template I like that fills my full 1680 x 1050 display and doesn't look like shit on the laptop's 1024 display!