The remarkable performance of Large Language Models (LLMs) gives rise to``LLM-as-a-Judge'', where LLMs are employed as evaluators for complex tasks. Moreover, it has been widely adopted across fields such as Natural Language Processing (NLP), preference learning, and various specific domains. However, there are various biases within LLM-as-a-Judge, which adversely affect the fairness and reliability of judgments. Current research on evaluating or mitigating bias in LLM-as-a-Judge predominantly focuses on comparison-based evaluations, while systematic investigations into bias in scoring-based evaluations remain limited. Therefore, we define scoring bias in LLM-as-a-Judge as the scores differ when scoring judge models are bias-related perturbed, and provide a well-designed framework to comprehensively evaluate scoring bias. We augment existing LLM-as-a-Judge benchmarks through data synthesis to construct our evaluation dataset and design multi-faceted evaluation metrics. Our experimental results demonstrate that the scoring stability of existing judge models is disrupted by scoring biases. Further exploratory experiments and discussions provide valuable insights into the design of scoring prompt templates and the mitigation of scoring biases on aspects such as score rubrics, score IDs, and reference answer selection.

Figure 1: Unsupervised Affordance Distillation (UAD) extracts affordance annotations from large pre-trained models and distills them into a task-conditioned affordance model, which is capable of predicting fine-grained affordance in open-world scenes with open-ended instructions, enabling diverse generalization properties in downstream policy learning. Abstract -- Understanding fine-grained object affordances is imperative for robots to manipulate objects in unstructured environments given open-ended task instructions. However, existing methods of visual affordance predictions often rely on manually annotated data or conditions only on a predefined set of tasks. We introduce Unsupervised Affordance Distillation (UAD), a method for distilling affordance knowledge from foundation models into a task-conditioned affordance model without any manual annotations . Training only a lightweight task-conditioned decoder atop frozen features, UAD exhibits notable generalization to in-the-wild robotic scenes and to various human activities, despite only being trained on rendered objects in simulation. Using affordance provided by UAD as the observation space, we show an imitation learning policy that demonstrates promising generalization to unseen object instances, object categories, and even variations in task instructions after training on as few as 10 demonstrations. Understanding the affordances of objects underpins a robot's capability to perform purposeful interactions in unstructured environments [1-3]. Given an open-ended task instruction specified in natural language, a robot must first identify the action possibilities afforded by the environment based on its visual perception. In particular, this understanding should extend beyond objects or object parts to encompass fine-grained details down to the level of pixels. While learning visual affordances from manually annotated datasets with closed vocabulary has been extensively explored in the literature [4-9], scaling affordance learning to open-world scenarios conditioned on free-form task instructions remains a long-standing challenge. Vision-language models (VLMs) have demonstrated the ability to internalize world knowledge by pretraining on large-scale image-text datasets [10, 11]. Recent works also suggest that they encode affordance knowledge in the language space [12] (e.g., "handle should be grasped for opening drawers"). However, the effective grounding of this knowledge in the continuous spatial domain remains an open question. In contrast, self-supervised vision models [13, 14] provide general-purpose pixel-level features that capture low-level structures of objects.

This work stems from an observed limitation of text encoders: embeddings may not be able to recognize fine-grained entities or events within encoded semantics, resulting in failed retrieval even in simple cases. To examine such behaviors, we first introduce a new evaluation dataset, CapRetrieval, in which passages are image captions and queries are phrases targeting entity or event concepts in diverse forms. Zero-shot evaluation suggests that encoders often struggle with these fine-grained matching, regardless of training sources or model size. Aiming for enhancement, we proceed to finetune encoders with our proposed data generation strategies, enabling a small 0.1B encoder to outperform the state-of-the-art 7B model. Within this process, we further uncover the granularity dilemma, a challenge for embeddings to capture fine-grained salience while aligning with overall semantics. Our dataset, code and models in this work are publicly released at https://github.com/lxucs/CapRetrieval.

Large Vision-Language Models (LVLMs) have made remarkable strides in multimodal tasks such as visual question answering, visual grounding, and complex reasoning. However, they remain limited by static training data, susceptibility to hallucinations, and inability to verify claims against up-to-date, external evidence, compromising their performance in dynamic real-world applications. Retrieval-Augmented Generation (RAG) offers a practical solution to mitigate these challenges by allowing the LVLMs to access large-scale knowledge databases via retrieval mechanisms, thereby grounding model outputs in factual, contextually relevant information. Here in this paper, we conduct the first systematic dissection of the multimodal RAG pipeline for LVLMs, explicitly investigating (1) the retrieval phase: on the modality configurations and retrieval strategies, (2) the re-ranking stage: on strategies to mitigate positional biases and improve the relevance of retrieved evidence, and (3) the generation phase: we further investigate how to best integrate retrieved candidates into the final generation process. Finally, we extend to explore a unified agentic framework that integrates re-ranking and generation through self-reflection, enabling LVLMs to select relevant evidence and suppress irrelevant context dynamically. Our full-stack exploration of RAG for LVLMs yields substantial insights, resulting in an average performance boost of 5% without any fine-tuning.

This paper investigates how Large Language Models (LLMs) represent non-English tokens -- a question that remains underexplored despite recent progress. We propose a lightweight intervention method using representation steering, where a learned vector is added to the residual stream at a single model layer to enhance multilingual performance. Through extensive experiments across seven competitive baselines -- including prompt optimization, supervised fine-tuning (SFT), in-context learning, cross-lingual transfer, and translation-based methods-we show that our approach consistently outperforms most alternatives. In particular, it achieves performance on par with production-grade translation systems while requiring far fewer resources. We further explore the complementarity between our method and SFT, demonstrating that steering offers a direct, efficient way to realign internal representations. These findings underscore the potential of activation-level interventions as a powerful tool for improving the multilingual capabilities of LLMs.

Guiding biological systems toward desired states, such as morphogenetic outcomes, remains a fundamental challenge with far-reaching implications for medicine and synthetic biology. While large language models (LLMs) have enabled natural language as an interface for interpretable control in AI systems, their use as mediators for steering biological or cellular dynamics remains largely unexplored. In this work, we present a functional pipeline that translates natural language prompts into spatial vector fields capable of directing simulated cellular collectives. Our approach combines a large language model with an evolvable neural controller (Prompt-to-Intervention, or P2I), optimized via evolutionary strategies to generate behaviors such as clustering or scattering in a simulated 2D environment. We demonstrate that even with constrained vocabulary and simplified cell models, evolved P2I networks can successfully align cellular dynamics with user-defined goals expressed in plain language. This work offers a complete loop from language input to simulated bioelectric-like intervention to behavioral output, providing a foundation for future systems capable of natural language-driven cellular control.

--Large language models (LLMs) are increasingly deployed through open-source and commercial frameworks, enabling individuals and organizations to self-host advanced LLM capabilities. As LLM deployments become prevalent, particularly in industry, ensuring their secure and reliable operation has become a critical issue. However, insecure defaults and miscon-figurations often expose LLM services to the public internet, posing serious security and system engineering risks. This study conducted a large-scale empirical investigation of public-facing LLM deployments, focusing on the prevalence of services, exposure characteristics, systemic vulnerabilities, and associated risks. Through internet-wide measurements, we identified 320,102 public-facing LLM services across 15 frameworks and extracted 158 unique API endpoints, categorized into 12 functional groups based on functionality and security risk. Our analysis found that over 40% of endpoints used plain HTTP, and over 210,000 endpoints lacked valid TLS metadata. API exposure was highly inconsistent: some frameworks, such as Ollama, responded to over 35% of unauthenticated API requests, with about 15% leaking model or system information, while other frameworks implemented stricter controls. We observed widespread use of insecure protocols, poor TLS configurations, and unauthenticated access to critical operations. These security risks, such as model leakage, system compromise, and unauthorized access, are pervasive and highlight the need for a secure-by-default framework and stronger deployment practices. Driven by renowned models like OpenAI's GPT series [33] and DeepSeek's open-source variant [9], large language models (LLMs) are rapidly gaining popularity and profoundly reshaping a wide range of applications. Once primarily confined to research labs and industrial environments, these models are now not only continuously deployed in-depth within the industry, but are also gradually opened to the wider public, promoting the vigorous development of self-hosted and open source deployment. The emergence of user-friendly tools and a vibrant community ecosystem [42], [43], [38] has enabled individual enthusiasts, small enterprises, and developers to independently deploy and customize powerful LLMs for a variety of personal and professional needs, such as creative writing and content creation [20], software development and maintenance [16], financial analysis and automated investment assistance [48], and personal productivity tools, significantly enriching their daily digital experiences.

On Tuesday, OpenAI published a blog post titled "Helping people when they need it most," that included sections on "What ChatGPT is designed to do," as well as "Where our systems can fall short, why, and how we're addressing" and the company's plans moving forward. It noted that it is working to strengthen safeguards for longer interactions. The complaint was filed by the Edelson PC law firm and the Tech Justice Law Project. The latter has been involved in a similar lawsuit against a different artificial intelligence company, Character.AI, in which Florida mother Megan Garcia claimed that one of the company's AI companions was responsible for the suicide of her 14-year-old son, Sewell Setzer III. The persona, she said, sent messages of an emotionally and sexually abusive nature to Sewell, which she alleges led to his death. A federal judge in May rejected its argument regarding constitutional protections "at this stage.")

At least three artificial intelligence researchers have resigned from Meta's new superintelligence lab, just two months after CEO Mark Zuckerberg first announced the initiative. Two of the staffers have returned to OpenAI, where they both previously worked, after less than one-month stints at Meta, WIRED has confirmed. Ethan Knight worked at the ChatGPT maker earlier in his career but joined Meta from Elon Musk's xAI. A third researcher, Rishabh Agarwal, announced publicly on Monday he was leaving Meta's lab as well. He joined the tech giant in April to work on generative AI projects before switching to a role at Meta Superintelligence Labs (MSL), according to his LinkedIn profile.

Training models with synthetic data presents both a danger and a boon to artificial intelligence (AI). While some groups have aggressively pursued the use of model-generated data to train successors for greater accuracy and generalization, others have warned about the risks posed by AI ingesting its own output. The two views are not at odds. The question is when and where things go wrong. On the negative side, a flurry of papers published since 2021 have argued that, as the datasets used to pretrain foundation models incorporate more and more auto-generated data mined from the Internet, performance degrades and the models start to "unlearn" skills.