Extracting Keywords From Short Text - The Ape Machine

Many years ago, the first time I ever needed to get to the most important keywords in a short sentence, I was developing some software for a company of psychologists who were making a platform that had a module which needed related sentences to be pulled from a database of linguistic challenges. The idea was to match the sentence to be retrieved with the keywords of the previous sentence, and as I knew very little about machine learning at the time, I went with my own naive solution to calculate the Scrabble score of each word in a sentence, and taking the top Nth scoring words from those results and assume them to be the keywords. This actually works surprisingly well, were it not for some stop-words that score quite high in Scrabble. Of course, you can just deploy a stop-list into this method, and things will get solved, which even is quite scalable, since any language only containes so many words you would like filtered out, and you only have to define this list once. The next implementation was Rapid Automated Keyword Extraction, which actually only works a little better than the previous method, and still makes use of a stop-list.