Language models demonstrate remarkable capacity to generalize representations learned in one modality to downstream tasks in other modalities. Can we trace this ability to individual neurons? We study the case where a frozen text transformer is augmented with vision using a self-supervised visual encoder and a single linear projection learned on an image-to-text task. Outputs of the projection layer are not immediately decodable into language describing image content; instead, we find that translation between modalities occurs deeper within the transformer. We introduce a procedure for identifying "multimodal neurons" that convert visual representations into corresponding text, and decoding the concepts they inject into the model's residual stream. In a series of experiments, we show that multimodal neurons operate on specific visual concepts across inputs, and have a systematic causal effect on image captioning.

Nick Cammarata†: Drew the connection between multimodal neurons in neural networks and multimodal neurons in the brain, which became the overall framing of the article. Created the conditional probability plots (regional, Trump, mental health), labeling more than 1500 images, discovered that negative pre-ReLU activations are often interpretable, and discovered that neurons sometimes contain a distinct regime change between medium and strong activations. Wrote the identity section and the emotion sections, building off Gabriel's discovery of emotion neurons and discovering that "complex" emotions can be broken down into simpler ones. Edited the overall text of the article and built infrastructure allowing the team to collaborate in Markdown with embeddable components.

We've discovered neurons in CLIP that respond to the same concept whether presented literally, symbolically, or conceptually. This may explain CLIP's accuracy in classifying surprising visual renditions of concepts, and is also an important step toward understanding the associations and biases that CLIP and similar models learn. Fifteen years ago, Quiroga et al. discovered that the human brain possesses multimodal neurons. These neurons respond to clusters of abstract concepts centered around a common high-level theme, rather than any specific visual feature. The most famous of these was the "Halle Berry" neuron, a neuron featured in both Scientific American and The New York Times, that responds to photographs, sketches, and the text "Halle Berry" (but not other names).

Researchers from machine learning lab OpenAI have discovered that their state-of-the-art computer vision system can be deceived by tools no more sophisticated than a pen and a pad. As illustrated in the image above, simply writing down the name of an object and sticking it on another can be enough to trick the software into misidentifying what it sees. "We refer to these attacks as typographic attacks," write OpenAI's researchers in a blog post. "By exploiting the model's ability to read text robustly, we find that even photographs of hand-written text can often fool the model." They note that such attacks are similar to "adversarial images" that can fool commercial machine vision systems, but far simpler to produce.

Elon Musk-founded startup OpenAI has discovered multimodal neurons in an Artificial Intelligence system that works just like some neurons work in the human brain. The revelation comes 15 years after the discovery that the human brain possesses multimodal neurons that respond to clusters of abstract concepts centered around a common high-level theme, rather than any specific visual feature. The most famous of these was the "Halle Berry" neuron that responds to photographs, sketches, and the text "Halle Berry" -- but not other names. Two months ago, OpenAI announced a neural network called CLIP which efficiently learns visual concepts from natural language supervision. CLIP can be applied to any visual classification benchmark by simply providing the names of the visual categories to be recognized. It is trained to recognize and people and objects within abstract contexts -- sketches, cartoons, and even statues of the objects.

Artificial Intelligence researchers at Open AI, a startup founded by Elon Musk, have discovered neurons within an AI system that have only previously been seen in the human brains. According to a blog post, researchers uncovered what is referred to by neuroscientists as a'multimodal neuron', within the murky inner workings of one of its most advanced neural networks. The researchers said that the discovery was made using a general-purpose vision called CLIP, which trains itself on complex datasets to recognise objects and people within abstracts, such as cartoons or statues. We've found that our latest vision model, CLIP, contains neurons that connect images, drawings and text about related concepts. The company said that they have discovered neurons in CLIP that respond to the same concept whether presented literally, symbolically, or conceptually.