In February 2024, it was widely reported that Reddit had signed a deal with Google, permitting Google to use Reddit’s content to train its AI models at a cost of $60 million per year. Initially seen as a licensing agreement for AI training, the deal appears to have expanded beyond its original scope.
According to 404 Media, the agreement has led to Google being the only search engine allowed to index Reddit’s new posts and content. This action underscores Google’s potentially anti-competitive behavior in the search engine market. As a result, Google is directing significant traffic to Reddit, while alternative search engines like Bing, DuckDuckGo, Mojeek, and Qwant are unable to access Reddit’s latest posts.
Search Engine Land has confirmed with Microsoft that Reddit is blocking all search engines except Google. According to the report, Reddit updated its “robots.txt” file on July 1, 2024, to prevent many search engines and AI bots from accessing its content. Notably, Google is the only major search engine permitted to crawl Reddit’s content.
I tried to locate new Reddit posts on Bing and DuckDuckGo, but they only showed posts from a month ago. This indicates that search engines are adhering to the “robots.txt” standard and have ceased crawling new Reddit content.
This situation raises concerns about Google’s monopolistic practices. Google is already under investigation by the US Department of Justice for antitrust issues, and this development could potentially exacerbate the situation for the company.
0 Comments