Google’s Gary Illyes answered a question about the non-search features that the GoogleOther crawler supports, then added a caution about the consequences of blocking GoogleOther.
What Is GoogleOther?
GoogleOther is a generic crawler created by Google for the various purposes that fall outside of those of bots that specialize for Search, Ads, Video, Images, News, Desktop and Mobile. It can be used by internal teams at Google for research and development in relation to various products.
The official description of GoogleOther is:
“GoogleOther is the generic crawler that may be used by various product teams for fetching publicly accessible content from sites. For example, it may be used for one-off crawls for internal research and development.”
Something that may be surprising is that there are actually three kinds of GoogleOther crawlers.
Three Kinds Of GoogleOther Crawlers
- GoogleOther
Generic crawler for public URLs - GoogleOther-Image
Optimized to crawl public image URLs - GoogleOther-Video
Optimized to crawl public video URLs
All three GoogleOther crawlers can be used for research and development purposes. That’s just one purpose that Google publicly acknowledges that all three versions of GoogleOther could be used for.
What Non-Search Features Does GoogleOther Support?
Google doesn’t say what specific non-search features GoogleOther supports, probably because it doesn’t really “support” a specific feature. It exists for research and development crawling which could be in support of a new product or an improvement in a current product, it’s a highly open and generic purpose.
This is the question asked that Gary narrated:
“What non-search features does GoogleOther crawling support?”