Understanding Transformer-based models has attracted significant attention, as they lie at the heart of recent technological advances across machine learning. While most interpretability methods rely on running models over inputs, recent work has shown that a zero-pass approach, where parameters are interpreted directly without a forward/backward pass is feasible for some Transformer parameters, and for two-layer attention networks. In this work, we present a theoretical analysis where all parameters of a trained Transformer are interpreted by projecting them into the embedding space, that is, the space of vocabulary items they operate on. We derive a simple theoretical framework to support our arguments and provide ample evidence for its validity. First, an empirical analysis showing that parameters of both pretrained and fine-tuned models can be interpreted in embedding space. Second, we present two applications of our framework: (a) aligning the parameters of different models that share a vocabulary, and (b) constructing a classifier without training by ``translating'' the parameters of a fine-tuned classifier to parameters of a different model that was only pretrained. Overall, our findings open the door to interpretation methods that, at least in part, abstract away from model specifics and operate in the embedding space only.

In 2022, we were presented with several stunning developments in artificial intelligence (AI). Some believe that these advances push the limits of what we have now (narrow AI) towards the holy grail of artificial general intelligence (a machine that can mimic the thinking and problem-solving capacities of humans but faster and more accurately). Among the many developments in 2022, four breakthroughs are of note and will be significant in 2023 and beyond both within the discussions on responsible design development and AI use and in the transformative power they have for our societies. First came DALL-E, the AI that can create pictures from language prompts. Many of us enjoyed playing with the tool and embracing the ability it gave to us to design in new ways. Others worried about AI taking over our human creativity.

Whilst I understand that progress in technologies can scare people, especially when there's no real limitations to the art-of-the-possible at the present time, this doesn't mean by any stretch of the imagination that every individual, working across every industry, at every level is going to be superseded by some form of AI. In actual fact, this is fear of the unknown! Each and every technological advance raises questions. This is why ensuring the creation of ethical or responsible AI capabilities is key. Instead of scare-mongering, companies and individuals need to embrace #ArtificialIntelligence (Tweet this) and actively let AI shape or suggest ways in which this could shape our work. There are a variety of repetitive tasks in my daily/weekly/monthly job which I hope #AI technologies will take away from me and allow me to have more time to spend productively!