r/MachineLearning - [D] How do we extract features from an LSTM Language Model

I recently read the "Learning to generate reviews and discovering sentiment" paper by OPENAI and found it to be super cool. But I could not understand how they are using the language model as feature extractor. Suppose we have 150 characters in a review, how do we extract features from these 150 characters when our input is 64 characters at a time.