resize
impacts limitations
Broader Impacts NaViT enables training of vision transformers on variable size inputs, which has a profound impact on advancing adaptive computation research. By training models to handle various input size, we can explore adaptive computation techniques that dynamically adjust the computational resources based on the specific requirements of a given input. This flexibility opens up new avenues for implementing ideas that aim at adjusting allocation of compute and improving efficiency in vision tasks per input. Furthermore, NaViT computational efficiency unlocks the potential for scaling up pre-training of vision models. With the ability to handle different resolutions, models can effectively tackle more complex and diverse visual data, allowing for the development of larger and more powerful vision models.
)V. (2) MSA is constructed based on Attention by split the channels ofQ,K and V into h groups with each group apart ofqueries, keys,and valuesQi,Ki RN
F s,iBs,i, n = 1,2,...,S, (10) where F s is the support features extracted by a pretrained ViT. Inspired by the multiple-object tracking within a single framework [21], in which different objects are represented by various identifications (i.e., learnable vectors) for simultaneously tracking, we add extra learnable tokens tothemeanfeatures formorediscriminativeprompts.
VACoT: Rethinking Visual Data Augmentation with VLMs
Xu, Zhengzhuo, Sun, Chong, Du, SiNan, Li, Chen, Lyu, Jing, Yuan, Chun
While visual data augmentation remains a cornerstone for training robust vision models, it has received limited attention in visual language models (VLMs), which predominantly rely on large-scale real data acquisition or synthetic diversity. Consequently, they may struggle with basic perception tasks that conventional models handle reliably. Given the substantial cost of pre-training and fine-tuning VLMs, continue training on augmented data yields limited and diminishing returns. In this paper, we present Visual Augmentation Chain-of-Thought (V ACoT), a framework that dynamically invokes image augmentations during model inference. By incorporating post-hoc transformations such as denoising, VACoT substantially improves robustness on challenging and out-of-distribution inputs, especially in OCR-related adversarial scenarios. Distinct from prior approaches limited to local cropping, VACoT integrates a structured collection of general visual augmentations, broadening the query image views while reducing training complexity and computational overhead with efficient agentic reinforcement learning. W e propose a conditional reward scheme that encourages necessary augmentation while penalizing verbose responses, ensuring concise and effective reasoning in perception tasks. W e demonstrate the superiority of VACoT with extensive experiments on 13 perception benchmarks and further introduce AdvOCR to highlight the generalization benefits of post-hoc visual augmentations in adversarial scenarios.
Competitive Programming with Large Reasoning Models
OpenAI, null, :, null, El-Kishky, Ahmed, Wei, Alexander, Saraiva, Andre, Minaev, Borys, Selsam, Daniel, Dohan, David, Song, Francis, Lightman, Hunter, Clavera, Ignasi, Pachocki, Jakub, Tworek, Jerry, Kuhn, Lorenz, Kaiser, Lukasz, Chen, Mark, Schwarzer, Max, Rohaninejad, Mostafa, McAleese, Nat, contributors, o3, Mรผrk, Oleg, Garg, Rhythm, Shu, Rui, Sidor, Szymon, Kosaraju, Vineet, Zhou, Wenda
We show that reinforcement learning applied to large language models (LLMs) significantly boosts performance on complex coding and reasoning tasks. Additionally, we compare two general-purpose reasoning models - OpenAI o1 and an early checkpoint of o3 - with a domain-specific system, o1-ioi, which uses hand-engineered inference strategies designed for competing in the 2024 International Olympiad in Informatics (IOI). We competed live at IOI 2024 with o1-ioi and, using hand-crafted test-time strategies, placed in the 49th percentile. Under relaxed competition constraints, o1-ioi achieved a gold medal. However, when evaluating later models such as o3, we find that o3 achieves gold without hand-crafted domain-specific strategies or relaxed constraints. Our findings show that although specialized pipelines such as o1-ioi yield solid improvements, the scaled-up, general-purpose o3 model surpasses those results without relying on hand-crafted inference heuristics. Notably, o3 achieves a gold medal at the 2024 IOI and obtains a Codeforces rating on par with elite human competitors. Overall, these results indicate that scaling general-purpose reinforcement learning, rather than relying on domain-specific techniques, offers a robust path toward state-of-the-art AI in reasoning domains, such as competitive programming.
This new Windows feature offers gaming advice while you play
If you've ever become "stuck" in a game, the new Microsoft Edge Game Assist for Windows offers a pretty convenient way to help yourself out: Simply look up a hint while playing. Microsoft debuted this new system as a beta during the holidays last year. Now, it's part of Microsoft Edge itself, provided that you have the current version, Edge 132 or higher. For those of you who want a "pure" gaming experience, don't worry. Edge Game Assist doesn't just launch itself, and it actually seems rather helpful when you invoke it.