Google real-time search accelerating crawls?
Google's recently launched real-time search functionality, which pulls data from a number of real time data sources, including the now publicly accessible Twitter firehose, is enhancing search functionality across the board. It seems that hyperlinks contained inside the microblog posts are being immediately indexed, and in turn is speeding up the time it takes for new content to appear not just in the live feed, but in the actual crawled index as well. Try doing some searches on Google for the title of any new blog posting from a major source, and note that the crawled index lists the post within minutes at most.
The real time feed could cause far reaching benefits for all of Google's services over the next few months as the big G continues their roll out. It's a safe bet that as this technology matures in the coming months we'll see further improvements that are thanks to the accessibility of real time data.
Jeff dePascale is a web and mobile applications professional based in Fairfield County, CT.
