Today, Google announced that they will no longer be crawling news sites with Googlebot-News and instead will crawl news sites with Googlebot, the same bot that crawls sites for web search. However, you can still block your content from being indexed in Google News by disallowing Googlebot-News in robots.txt or using a meta robots tag.
Blocking Content from Google News
Seem confusing? On the one hand, it’s not at all.
If you want Google to index your content in both web search and News (if you are a Google News publisher), then you don’t need to do anything. Google will keep crawling as it always has, but if you look at your server logs, you’ll only see entries for Googlebot rather than entries for both Googlebot and Googlebot-News.
If you want to keep your content out of Google News, you can keeping using the Disallow directive in robots.txt (or meta robots tag) to block Googlebot-News. Even though Google will now crawl as Googlebot rather than Googlebot-News, they’ll still respect the Googleb0t-News robots.txt directive.
Presumably, you can also allow your content in Google News and block it from web search by using a combination of a Disallow directive for Googlebot and an Allow directive for Googlebot-News (although I’m still trying to get verification of that from Google).
Gathering Data about How Your Site Is Crawled
On the other hand, this change makes things a lot more confusing if you’re using data to understand how your site is crawled and make improvements.
for more details visit search engine land
About Me
- seo services india
- Atechnocrat is a website design Development services & SEO service company, dedicated to the all SEO community and professionals. This comany is associated with Atechnocrat.com, a web services company which provides SEO services, web application development, website design services, open source development, web development services and many other web services to its clients and customers in UK.
No comments:
Post a Comment