High-Return Data Science: Modernizing / Automating Digital Publishing (Case Study)

Here I put together a number of new data science techniques to solve a real life problem: identifying good articles to write and publish (or to harvest and re-post) on a website, and re-tweet them with the optimum frequency, given a specific audience. The focus is on scoring articles based on selected features (keywords in subject line, author, channel and many more), feature selection, data generation and harvesting to solve the problem, automatically categorizing and tagging articles using indexation algorithms, and predicting the lifetime value and total page views of an article. We also discuss bucketisation, and use both hidden decision trees and jackknife regression for scoring articles. The methodology applies to many contexts, not just digital publishing. It applies to situations in which a lot of unstructured text data needs to be processed (categorized and scored using natural language processing methods).