METEOR: Learning Memory and Time Efficient Representations from Multi-modal Data Streams