![Reddit Limits Search Engine Access, Google Remains Exception Reddit Limits Search Engine Access, Google Remains Exception](https://www.searchenginejournal.com/wp-content/uploads/2024/07/img_2024-07-26-150340-445.jpeg)
Reddit has recently tightened its grip on who can access its content, blocking major search engines from indexing recent posts and comments.
This move has sparked discussions in the SEO and digital marketing communities about the future of content accessibility and AI training data.
What’s Happening?
First reported by 404 Media, Reddit updated its robots.txt file, preventing most web crawlers from accessing its latest content.
Google, however, remains an exception, likely due to a $60 million deal that allows the search giant to use Reddit’s content for AI training.
Brent Csutoras, founder of Search Engine Journal, offers some context:
“Since taking on new investors and starting their pathway to IPO, Reddit has moved away from being open-source and allowing anyone to scrape their content and use their APIs without paying.”
The Google Exception
Currently, Google is the only major search engine able to display recent Reddit results when users search with “site:reddit.com.”
This exclusive access sets Google apart from competitors like Bing and DuckDuckGo.
Why This Matters
For users who rely on appending “Reddit” to their searches to find human-generated answers, this change means they’ll be limited to using Google or search engines that pull from Google’s index.