aoc
An Adaptive Orthogonal Convolution Scheme for Efficient and Flexible CNN Architectures
Boissin, Thibaut, Mamalet, Franck, Fel, Thomas, Picard, Agustin Martin, Massena, Thomas, Serrurier, Mathieu
Orthogonal convolutional layers are the workhorse of multiple areas in machine learning, such as adversarial robustness, normalizing flows, GANs, and Lipschitzconstrained models. Their ability to preserve norms and ensure stable gradient propagation makes them valuable for a large range of problems. Despite their promise, the deployment of orthogonal convolution in large-scale applications is a significant challenge due to computational overhead and limited support for modern features like strides, dilations, group convolutions, and transposed convolutions.In this paper, we introduce AOC (Adaptative Orthogonal Convolution), a scalable method for constructing orthogonal convolutions, effectively overcoming these limitations. This advancement unlocks the construction of architectures that were previously considered impractical. We demonstrate through our experiments that our method produces expressive models that become increasingly efficient as they scale. To foster further advancement, we provide an open-source library implementing this method, available at https://github.com/thib-s/orthogonium.
AOC played video game with Walz as constituents protested against prostitution in her 'Third World' district
More than two dozen prostitutes line a Queens New York City street soliciting sex. At the exact time Rep. Alexandria Ocasio-Cortez, D-N.Y., was live-streaming her "Madden" NFL video game session with vice presidential candidate Tim Walz, on Twitch, her constituents were taking to the streets to protest rampant illegal prostitution and crime in the neighborhood she represents. The progressive "Squad" member was slammed by fellow Democrat politician Hiram Monserrate for playing the video game on the streaming service Sunday afternoon while residents from her district held a rally calling for their community to be cleaned up. "We need advocates not gamers," Monserrate, a former New York state senator who is running for State Assembly, told Fox News Digital. The Queens neighborhood is well known as a "Red Light" district, with some residents comparing the unsanitary and seedy conditions to a "Third World" country.
PECC: Problem Extraction and Coding Challenges
Haller, Patrick, Golde, Jonas, Akbik, Alan
Recent advancements in large language models (LLMs) have showcased their exceptional abilities across various tasks, such as code generation, problem-solving and reasoning. Existing benchmarks evaluate tasks in isolation, yet the extent to which LLMs can understand prose-style tasks, identify the underlying problems, and then generate appropriate code solutions is still unexplored. Addressing this gap, we introduce PECC, a novel benchmark derived from Advent Of Code (AoC) challenges and Project Euler, including 2396 problems. Unlike conventional benchmarks, PECC requires LLMs to interpret narrative-embedded problems, extract requirements, and generate executable code. A key feature of our dataset is the complexity added by natural language prompting in chat-based evaluations, mirroring real-world instruction ambiguities. Results show varying model performance between narrative and neutral problems, with specific challenges in the Euler math-based subset with GPT-3.5-Turbo passing 50% of the AoC challenges and only 8% on the Euler problems. By probing the limits of LLMs' capabilities, our benchmark provides a framework to monitor and assess the subsequent progress of LLMs as a universal problem solver.
Is there really a Citation Age Bias in NLP?
Citations are a key ingredient of scientific research to relate a paper to others published in the community. Recently, it has been noted that there is a citation age bias in the Natural Language Processing (NLP) community, one of the currently fastest growing AI subfields, in that the mean age of the bibliography of NLP papers has become ever younger in the last few years, leading to `citation amnesia' in which older knowledge is increasingly forgotten. In this work, we put such claims into perspective by analyzing the bibliography of $\sim$300k papers across 15 different scientific fields submitted to the popular preprint server Arxiv in the time period from 2013 to 2022. We find that all AI subfields (in particular: cs.AI, cs.CL, cs.CV, cs.LG) have similar trends of citation amnesia, in which the age of the bibliography has roughly halved in the last 10 years (from above 12 in 2013 to below 7 in 2022), on average. Rather than diagnosing this as a citation age bias in the NLP community, we believe this pattern is an artefact of the dynamics of these research fields, in which new knowledge is produced in ever shorter time intervals.
Accountability in Offline Reinforcement Learning: Explaining Decisions with a Corpus of Examples
Sun, Hao, Hรผyรผk, Alihan, Jarrett, Daniel, van der Schaar, Mihaela
Learning controllers with offline data in decision-making systems is an essential area of research due to its potential to reduce the risk of applications in real-world systems. However, in responsibility-sensitive settings such as healthcare, decision accountability is of paramount importance, yet has not been adequately addressed by the literature. This paper introduces the Accountable Offline Controller (AOC) that employs the offline dataset as the Decision Corpus and performs accountable control based on a tailored selection of examples, referred to as the Corpus Subset. AOC operates effectively in low-data scenarios, can be extended to the strictly offline imitation setting, and displays qualities of both conservation and adaptability. We assess AOC's performance in both simulated and real-world healthcare scenarios, emphasizing its capability to manage offline control tasks with high levels of performance while maintaining accountability.
Attention Option-Critic
Chunduru, Raviteja, Precup, Doina
Temporal abstraction in reinforcement learning is the ability of an agent to learn and use high-level behaviors, called options. The option-critic architecture provides a gradient-based end-to-end learning method to construct options. We propose an attention-based extension to this framework, which enables the agent to learn to focus different options on different aspects of the observation space. We show that this leads to behaviorally diverse options which are also capable of state abstraction, and prevents the degeneracy problems of option domination and frequent option switching that occur in option-critic, while achieving a similar sample complexity. We also demonstrate the more efficient, interpretable, and reusable nature of the learned options in comparison with option-critic, through different transfer learning tasks. Experimental results in a relatively simple four-rooms environment and the more complex ALE (Arcade Learning Environment) showcase the efficacy of our approach.
An AI saw a cropped photo of AOC. It autocompleted her wearing a bikini.
Ryan Steed, a PhD student at Carnegie Mellon University, and Aylin Caliskan, an assistant professor at George Washington University, looked at two algorithms: OpenAI's iGPT (a version of GPT-2 that is trained on pixels instead of words) and Google's SimCLR. While each algorithm approaches learning images differently, they share an important characteristic--they both use completely unsupervised learning, meaning they do not need humans to label the images. This is a relatively new innovation as of 2020. Previous computer-vision algorithms mainly used supervised learning, which involves feeding them manually labeled images: cat photos with the tag "cat" and baby photos with the tag "baby." But in 2019, researcher Kate Crawford and artist Trevor Paglen found that these human-created labels in ImageNet, the most foundational image data set for training computer-vision models, sometimes contain disturbing language, like "slut" for women and racial slurs for minorities.
AOC playing 'Among Us' shouldn't surprise you. Streams are a beloved pastime.
Video games also encompass and utilize every single other medium, like art, design, text and music, which also explain their viewable appeal. To watch a beloved streamer go through "The Last of Us Part II" is like watching a friend play a famous role in a film or stage play, yet somehow occupying the perspectives of both character and the person they are. And for many viewers, they've played this same role themselves by playing the game. It's a great way to size up a person, get a read of who they are, and feel at ease with them. Video games can be sports, or they can be performance art, or a comedy show.