Agile Tortoise
Greg Pierce’s blog
« Wireless network connection is not connected Sprint PCS problems »
Yahoo’s image bot
Yahoo’s image search bot doesn’t pay attention to robots.txt files, apparently. Our family site has been a “no index” zone for a long time, but the images from it are appearing in Yahoo image search results. I haven’t dug through the logs to see when they index them, but I’m seeing the referrers, so I decided to make the site membership-only.
I don’t know if that’s a grey area in the spec or not, since robots.txt was designed for text indexers, but I know google and most others just go away and don’t consider the images for their searchs if they are told to ignore an entire site by the robots.txt at the root level.
