Goto

Collaborating Authors

 serpent


Did a Chatbot Write a Prize-Winning Story? Does It Matter?

The New Yorker

Did a Chatbot Write a Prize-Winning Story? If the possibility that one or more of the winners of the Commonwealth Short Story Prize was A.I.-generated chills us, it may be because of what it reveals about human writing. In early May, the Commonwealth Foundation announced the five regional winners for its influential Short Story Prize, which recognizes unpublished short fiction. One of the awardees, a Trinidadian writer named Jamir Nazir, was accused of A.I.-assisted cheating by a broad array of social-media users who seized upon his story's synthetic tics, glitchy metaphors, and general unreadability. "Maybe it was a name; maybe rain took a shape and decided to keep it.")


Embryology of a Language Model

arXiv.org Artificial Intelligence

Understanding how language models develop their internal computational structure is a central problem in the science of deep learning. While susceptibilities, drawn from statistical physics, offer a promising analytical tool, their full potential for visualizing network organization remains untapped. In this work, we introduce an embryological approach, applying UMAP to the susceptibility matrix to visualize the model's structural development over training. Our visualizations reveal the emergence of a clear ``body plan,'' charting the formation of known features like the induction circuit and discovering previously unknown structures, such as a ``spacing fin'' dedicated to counting space tokens. This work demonstrates that susceptibility analysis can move beyond validation to uncover novel mechanisms, providing a powerful, holistic lens for studying the developmental principles of complex neural networks.


Cluster automata

arXiv.org Artificial Intelligence

Clustered Moore automata (CMA) are subsequen-tial Moore transducers whose states can contain smaller CMA that operate on a faster timescale, subject to an Artinian limitation.


SeRpEnt: Selective Resampling for Expressive State Space Models

arXiv.org Artificial Intelligence

State Space Models (SSMs) have recently enjoyed a rise to prominence in the field of deep learning for sequence modeling, especially as an alternative to Transformers. Their success stems from avoiding two well-known drawbacks of attention-based models: quadratic complexity with respect to the sequence length and inability to model long-range dependencies. The SSM variant Mamba has demonstrated performance comparable to Transformers without any form of attention, thanks to the use of a selective mechanism for the state parameters. Selectivity, however, is only evaluated empirically and the reasons of its effectiveness remain unclear. In this work, we show how selectivity is related to the sequence processing. Our analysis shows that selective time intervals in Mamba act as linear approximators of information. Then, we propose our SeRpEnt architecture, a SSM that further exploits selectivity to compress sequences in an information-aware fashion. It employs a resampling mechanism that aggregates elements based on their information content. Our empirical results in the Long Range Arena benchmark and other language modeling tasks show benefits of the SeRpEnt's resampling mechanism.


Serpent: Scalable and Efficient Image Restoration via Multi-scale Structured State Space Models

arXiv.org Artificial Intelligence

Image restoration is aimed at recovering a clean image from its degraded counterpart, encompassing crucial tasks such as superresolution [11, 22], deblurring [19, 27], inpainting [25, 14] and JPEG compression artifact removal [3]. End-to-end deep learning techniques that directly learn the mapping from corrupted images to their clean counterparts are the current state-of-the-art in most image recovery tasks. The careful design of such architectures has attracted considerable attention in recent years, and is crucial for the performance and efficiency of image restoration methods. Architectures composed of convolutional building blocks have achieved great success in a multitude of image restoration problems [15, 20] thanks to their compute efficiency. However, convolutional neural networks (CNNs) are limited in low-level vision tasks by two key weaknesses. First, convolutional filters are content-independent, that is different image regions are processed by the same filter. Second, convolutions have limited capability to model long-range dependencies due to the small size of kernels, requiring exceedingly deeper architectures to increase the receptive field. More recently, Transformer architectures such as the Vision Transformer [2], have shown enormous potential in a variety of vision problems, including dense prediction tasks such as image restoration [26, 23, 12, 28]. Vision Transformers split the image into non-overlapping patches, and process the patches in an embedded token representation.


The 10 Best Shows on Apple TV Right Now

WIRED

Slowly but surely Apple TV is finding its feet. The streaming service, which at launch we called "odd, angsty, and horny as hell," has evolved into a diverse library of dramas, documentaries, and comedies. It's also fairly cheap compared to services like Netflix--and Apple often throws in three free months when you buy a new iPhone, iPad, Mac, or Apple TV. Curious but don't know where to get started? Below are our picks for the best shows on the service.


SerpentAI/SerpentAI

#artificialintelligence

Serpent.AI is a simple yet powerful, novel framework to assist developers in the creation of game agents. The framework's raison d'être is first and foremost to provide a valuable tool for Machine Learning & AI research. It also turns out to be ridiculously fun to use as a hobbyist (and dangerously addictive; a fair warning)! The framework features a large assortment of supporting modules that provide solutions to commonly encountered scenarios when using video games as environments as well as CLI tools to accelerate development. It provides some useful conventions but is absolutely NOT opiniated about what you put in your agents: Want to use the latest, cutting-edge deep reinforcement learning algorithm?