NLP News Cypher

Once in a while, cool things happen, and this past week, the AdapterHub framework dropped. In the next evolution of NLP transfer learning, adapters deliver a new (and more modular) architecture. Oh, we assumed most of you would say "WTF are adapters?!" As a result, we were really excited to speak with AdapterHub's author Jonas Pfeiffer to get us up to speed on everything adapters and their framework: "Adapters are small modular units encapsulated within every layer of a transformer model, which learn to store task or language specific information. This is achieved by training *only* the newly introduced adapter weights, while keeping the rest of the pre-trained model fixed. The most fascinating concept about adapters is their modularity which opens up many possibilities of combining the knowledge from many adapters trained on a multitude of tasks. In order to make training adapters and subsequently sharing them as easy as possible, we have proposed the AdapterHub framework."