"You shouldn't have put your content on the internet if you didn't want it to be on the internet," Common Crawl's executive director says. Listen to more stories on the Noa app. T he Common Crawl Foundation is little known outside of Silicon Valley. For more than a decade, the nonprofit has been scraping billions of webpages to build a massive archive of the internet. This database--large enough to be measured in petabytes--is made freely available for research.

Chatbots are vying to become one of the cornerstones of the messaging world: using AI tools like natural language and machine learning, developers are hoping to tap into the popularity of chat apps as a medium of communication to explore new ways to help you get information, buy things, plan your life and more by letting you converse with intelligent computers instead of humans. In the latest release, presented today at the Hackathon at TC Disrupt in San Francisco, a chatbot is hoping to drop some literary knowledge on the world and create a fun way of getting answers to your most pressing questions about life. Shakespeare, as the bot is called, is a new messaging bot based on the works of the Bard of Avon. The developers have taken his poems and plays -- which are available open-source -- pulled out all of the most famous lines, and compiled them into a database. They then applied a natural language parser to index and understand the lines.