Large language models (LLMs) are aligned with human preferences by reinforcement learning from human feedback (RLHF). Effective data sampling is crucial for RLHF, as it determines the efficiency of model training, ensuring that models learn from the informative samples. To achieve better data generation, we propose a new sampling method called Preference-Guided Reflective Sampling (PRS). PRS frames the response generation as an optimization process to the explicitly specified user preference described in natural language. It employs a tree-based generation framework to enable an efficient sampling process, which guides the direction of generation through preference and better explores the sampling space with adaptive self-refinement. Notably, PRS can align LLMs to diverse preferences. We study preference-controlled text generation for instruction following and keyword-focused document summarization. Our findings indicate that PRS, across different LLM policies, generates training data with much higher rewards than strong baselines. PRS also excels in post-RL training.

In the downstream information retrieval process of DNA storage technology, specific hybridization techniques, such as Polymerase Chain Reaction (PCR) or magnetic bead separation, are commonly used to access data [1]. However, this technology faces several challenges, including high base error rates (insertions, deletions, substitutions, etc.) and the loss of storage sequences, which pose significant threats to the reliability of stored data [2]. To address these issues, clustering and alignment of sequencing data can be employed. A commonly used feature extraction method is based on k-mer frequency matrices, where the dimensionality of the extracted features increases exponentially with the value of k [3] [4] [5]. Therefore, selecting an appropriate dimensionality reduction technique becomes a critical challenge that needs to be addressed. This study aims to develop an adaptive classification model to identify the optimal dimensionality reduction method, thereby mitigating the curse of dimensionality caused by k-mer feature extraction and enhancing the effectiveness of K-means clustering in restoring the original sequence information. Specifically, among the numerous available algorithms, Principal Component Analysis (PCA) [6], t-distributed Stochastic Neighbor Embedding (t-SNE) [7], and Uniform Manifold Approximation and Projection (UMAP) [8] are particularly prominent in the fields of cell biology, bioinformatics, and data visualization [9]. This study addresses the challenge of selecting the appropriate dimensionality reduction method to mitigate the curse of dimensionality in K-means clustering.

Recent technological advancements have enhanced our ability to collect and analyze rich multimodal data (e.g., speech, video, and eye gaze) to better inform learning and training experiences. While previous reviews have focused on parts of the multimodal pipeline (e.g., conceptual models and data fusion), a comprehensive literature review on the methods informing multimodal learning and training environments has not been conducted. This literature review provides an in-depth analysis of research methods in these environments, proposing a taxonomy and framework that encapsulates recent methodological advances in this field and characterizes the multimodal domain in terms of five modality groups: Natural Language, Video, Sensors, Human-Centered, and Environment Logs. We introduce a novel data fusion category -- mid fusion -- and a graph-based technique for refining literature reviews, termed citation graph pruning. Our analysis reveals that leveraging multiple modalities offers a more holistic understanding of the behaviors and outcomes of learners and trainees. Even when multimodality does not enhance predictive accuracy, it often uncovers patterns that contextualize and elucidate unimodal data, revealing subtleties that a single modality may miss. However, there remains a need for further research to bridge the divide between multimodal learning and training studies and foundational AI research.

In Natural Language Processing (NLP), Large Language Models (LLMs) have demonstrated high text generation quality. However, in real-world applications, LLMs must meet increasingly complex requirements. Beyond avoiding misleading or inappropriate content, LLMs are also expected to cater to specific user needs, such as imitating particular writing styles or generating text with poetic richness. These varied demands have driven the development of Controllable Text Generation (CTG) techniques, which ensure that outputs adhere to predefined control conditions--such as safety, sentiment, thematic consistency, and linguistic style--while maintaining high standards of helpfulness, fluency, and diversity. This paper systematically reviews the latest advancements in CTG for LLMs, offering a comprehensive definition of its core concepts and clarifying the requirements for control conditions and text quality. We categorize CTG tasks into two primary types: content control and attribute control. The key methods are discussed, including model retraining, fine-tuning, reinforcement learning, prompt engineering, latent space manipulation, and decoding-time intervention. We analyze each method's characteristics, advantages, and limitations, providing nuanced insights for achieving generation control. Additionally, we review CTG evaluation methods, summarize its applications across domains, and address key challenges in current research, including reduced fluency and practicality. We also propose several appeals, such as placing greater emphasis on real-world applications in future research. This paper aims to offer valuable guidance to researchers and developers in the field. Our reference list and Chinese version are open-sourced at https://github.com/IAAR-Shanghai/CTGSurvey.

We are living in an age of protest. Although we have an excellent understanding of the factors that predict participation in protest, we understand little about the conditions that foster a sustained (versus transient) movement. How do interactions between supporters and authorities combine to influence whether and how people engage (i.e., using conventional or radical tactics)? This paper introduces a novel, theoretically-founded and empirically-informed agent-based model (DIMESim) to address these questions. We model the complex interactions between the psychological attributes of the protester (agents), the authority to whom the protests are targeted, and the environment that allows protesters to coordinate with each other -- over time, and at a population scale. Where an authority is responsive and failure is contested, a modest sized conventional movement endured. Where authorities repeatedly and incontrovertibly fail the movement, the population disengaged from action but evidenced an ongoing commitment to radicalism (latent radicalism).

The field of vision-language models (VLMs), which take images and texts as inputs and output texts, is rapidly evolving and has yet to reach consensus on several key aspects of the development pipeline, including data, architecture, and training methods. This paper can be seen as a tutorial for building a VLM. We begin by providing a comprehensive overview of the current state-of-the-art approaches, highlighting the strengths and weaknesses of each, addressing the major challenges in the field, and suggesting promising research directions for underexplored areas. We then walk through the practical steps to build Idefics3-8B, a powerful VLM that significantly outperforms its predecessor Idefics2-8B, while being trained efficiently, exclusively on open datasets, and using a straightforward pipeline. These steps include the creation of Docmatix, a dataset for improving document understanding capabilities, which is 240 times larger than previously available datasets. We release the model along with the datasets created for its training.

It has been a year with no major new console launches and where the industry has seen strikes and cuts with thousands of workers being laid off. The opening night of Gamescom is often an opportunity for a big shiny night to get fans all excited for the year ahead. Setting the stage for the next 12 months, here are the biggest things we found out from Europe's biggest gaming show in Germany. In a year when games became films, and films became games, the convention centre in Cologne saw a night all about the big trailers. This year, Borderlands has taken attention for its movie adaptation starring Cate Blanchett and Kevin Hart. That film received some of the year's harshest reviews, but that has not scuppered plans for a new game in the mainline series.

Kentucky Gov. Andy Beshear defended suggesting GOP vice-presidential nominee JD Vance go through a rape-induced abortion to understand the need for reproductive rights in an interview on MSNBC. There's really no other choice but to let Trump be Trump. While Kamala Harris rocked the DNC with an unexpected appearance and short speech to the rapturous crowd, Donald Trump continued to attack his new opponent, sometimes in odd ways. He is ignoring public advice from such close Republican allies as Lindsey Graham. "If you have a policy debate, he wins," the senator said on "Meet the Press."

Grounded Multimodal Named Entity Recognition (GMNER) is an emerging information extraction (IE) task, aiming to simultaneously extract entity spans, types, and corresponding visual regions of entities from given sentence-image pairs data. Recent unified methods employing machine reading comprehension or sequence generation-based frameworks show limitations in this difficult task. The former, utilizing human-designed queries, struggles to differentiate ambiguous entities, such as Jordan (Person) and off-White x Jordan (Shoes). The latter, following the one-by-one decoding order, suffers from exposure bias issues. We maintain that these works misunderstand the relationships of multimodal entities. To tackle these, we propose a novel unified framework named Multi-grained Query-guided Set Prediction Network (MQSPN) to learn appropriate relationships at intra-entity and inter-entity levels. Specifically, MQSPN consists of a Multi-grained Query Set (MQS) and a Multimodal Set Prediction Network (MSP). MQS explicitly aligns entity regions with entity spans by employing a set of learnable queries to strengthen intra-entity connections. Based on distinct intra-entity modeling, MSP reformulates GMNER as a set prediction, guiding models to establish appropriate inter-entity relationships from a global matching perspective. Additionally, we incorporate a query-guided Fusion Net (QFNet) to work as a glue network between MQS and MSP. Extensive experiments demonstrate that our approach achieves state-of-the-art performances in widely used benchmarks.

Automatic Program Repair (APR) endeavors to autonomously rectify issues within specific projects, which generally encompasses three categories of tasks: bug resolution, new feature development, and feature enhancement. Despite extensive research proposing various methodologies, their efficacy in addressing real issues remains unsatisfactory. It's worth noting that, typically, engineers have design rationales (DR) on solution-planed solutions and a set of underlying reasons-before they start patching code. In open-source projects, these DRs are frequently captured in issue logs through project management tools like Jira. This raises a compelling question: How can we leverage DR scattered across the issue logs to efficiently enhance APR? To investigate this premise, we introduce DRCodePilot, an approach designed to augment GPT-4-Turbo's APR capabilities by incorporating DR into the prompt instruction. Furthermore, given GPT-4's constraints in fully grasping the broader project context and occasional shortcomings in generating precise identifiers, we have devised a feedback-based self-reflective framework, in which we prompt GPT-4 to reconsider and refine its outputs by referencing a provided patch and suggested identifiers. We have established a benchmark comprising 938 issue-patch pairs sourced from two open-source repositories hosted on GitHub and Jira. Our experimental results are impressive: DRCodePilot achieves a full-match ratio that is a remarkable 4.7x higher than when GPT-4 is utilized directly. Additionally, the CodeBLEU scores also exhibit promising enhancements. Moreover, our findings reveal that the standalone application of DR can yield promising increase in the full-match ratio across CodeLlama, GPT-3.5, and GPT-4 within our benchmark suite. We believe that our DRCodePilot initiative heralds a novel human-in-the-loop avenue for advancing the field of APR.