The Company Quietly Funneling Paywalled Articles to AI Developers

"You shouldn't have put your content on the internet if you didn't want it to be on the internet," Common Crawl's executive director says. Listen to more stories on the Noa app. T he Common Crawl Foundation is little known outside of Silicon Valley. For more than a decade, the nonprofit has been scraping billions of webpages to build a massive archive of the internet. This database--large enough to be measured in petabytes--is made freely available for research.