As scaling large language models faces prohibitive costs, multi-agent systems emerge as Multi-Agent Systems (MAS) (Guo et al., 2024b) offer a a promising alternative, though challenged by promising alternative by coordinating multiple specialized static knowledge assumptions and coordination agents to achieve superior performance compared to individual inefficiencies. We introduce Knowledge-Aware systems while maintaining manageable computational Bayesian Bandits (KABB), a novel framework costs and budgets. Recent advances in MAS have led to that enhances multi-agent system coordination the development of several frameworks. For example, the through semantic understanding and dynamic Mixture of Agents (MoA) (Wang et al., 2024) employs multiple adaptation. The framework features three key LLMs as proposers to iteratively refine responses, with innovations: a three-dimensional knowledge distance a central aggregator delivering the final output. Although model for deep semantic understanding, a MoA has demonstrated robustness and scalability in deployment, dual-adaptation mechanism for continuous expert its computational cost scales linearly with the number optimization, and a knowledge-aware Thompson of agents, and significant redundancy and noise become a Sampling strategy for efficient expert selection.

Multimodal large language models have experienced rapid growth, and numerous different models have emerged. The interpretability of LVLMs remains an under-explored area. Especially when faced with more complex tasks such as chain-of-thought reasoning, its internal mechanisms still resemble a black box that is difficult to decipher. By studying the interaction and information flow between images and text, we noticed that in models such as LLaVA1.5, image tokens that are semantically related to text are more likely to have information flow convergence in the LLM decoding layer, and these image tokens receive higher attention scores. However, those image tokens that are less relevant to the text do not have information flow convergence, and they only get very small attention scores. To efficiently utilize the image information, we propose a new image token reduction method, Simignore, which aims to improve the complex reasoning ability of LVLMs by computing the similarity between image and text embeddings and ignoring image tokens that are irrelevant and unimportant to the text. Through extensive experiments, we demonstrate the effectiveness of our method for complex reasoning tasks. The paper's source code can be accessed from \url{https://github.com/FanshuoZeng/Simignore}.

Multiple clustering aims to discover various latent structures of data from different aspects. Deep multiple clustering methods have achieved remarkable performance by exploiting complex patterns and relationships in data. However, existing works struggle to flexibly adapt to diverse user-specific needs in data grouping, which may require manual understanding of each clustering. To address these limitations, we introduce Multi-Sub, a novel end-to-end multiple clustering approach that incorporates a multi-modal subspace proxy learning framework in this work. Utilizing the synergistic capabilities of CLIP and GPT-4, Multi-Sub aligns textual prompts expressing user preferences with their corresponding visual representations. This is achieved by automatically generating proxy words from large language models that act as subspace bases, thus allowing for the customized representation of data in terms specific to the user's interests. Our method consistently outperforms existing baselines across a broad set of datasets in visual multiple clustering tasks.

Data selection is essential for training deep learning models. An effective data sampler assigns proper sampling probability for training data and helps the model converge to a good local minimum with high performance. Previous studies in data sampling are mainly based on heuristic rules or learning through a huge amount of time-consuming trials. In this paper, we propose an automatic \textbf{swift sampler} search algorithm, \textbf{SS}, to explore automatically learning effective samplers efficiently. In particular, \textbf{SS} utilizes a novel formulation to map a sampler to a low dimension of hyper-parameters and uses an approximated local minimum to quickly examine the quality of a sampler. Benefiting from its low computational expense, \textbf{SS} can be applied on large-scale data sets with high efficiency. Comprehensive experiments on various tasks demonstrate that \textbf{SS} powered sampling can achieve obvious improvements (e.g., 1.5\% on ImageNet) and transfer among different neural networks. Project page: https://github.com/Alexander-Yao/Swift-Sampler.

We consider forecasting a single time series using high-dimensional predictors in the presence of a possible nonlinear forecast function. The sufficient forecasting (Fan et al., 2016) used sliced inverse regression to estimate lower-dimensional sufficient indices for nonparametric forecasting using factor models. However, Fan et al. (2016) is fundamentally limited to the inverse first-moment method, by assuming the restricted fixed number of factors, linearity condition for factors, and monotone effect of factors on the response. In this work, we study the inverse second-moment method using directional regression and the inverse third-moment method to extend the methodology and applicability of the sufficient forecasting. As the number of factors diverges with the dimension of predictors, the proposed method relaxes the distributional assumption of the predictor and enhances the capability of capturing the non-monotone effect of factors on the response. We not only provide a high-dimensional analysis of inverse moment methods such as exhaustiveness and rate of convergence, but also prove their model selection consistency. The power of our proposed methods is demonstrated in both simulation studies and an empirical study of forecasting monthly macroeconomic data from Q1 1959 to Q1 2016. During our theoretical development, we prove an invariance result for inverse moment methods, which make a separate contribution to the sufficient dimension reduction.

We consider forecasting a single time series when there is a large number of predictors and a possible nonlinear effect. The dimensionality was first reduced via a high-dimensional (approximate) factor model implemented by the principal component analysis. Using the extracted factors, we develop a novel forecasting method called the sufficient forecasting, which provides a set of sufficient predictive indices, inferred from high-dimensional predictors, to deliver additional predictive power. The projected principal component analysis will be employed to enhance the accuracy of inferred factors when a semi-parametric (approximate) factor model is assumed. Our method is also applicable to cross-sectional sufficient regression using extracted factors. The connection between the sufficient forecasting and the deep learning architecture is explicitly stated. The sufficient forecasting correctly estimates projection indices of the underlying factors even in the presence of a nonparametric forecasting function. The proposed method extends the sufficient dimension reduction to high-dimensional regimes by condensing the cross-sectional information through factor models. We derive asymptotic properties for the estimate of the central subspace spanned by these projection directions as well as the estimates of the sufficient predictive indices. We further show that the natural method of running multiple regression of target on estimated factors yields a linear estimate that actually falls into this central subspace. Our method and theory allow the number of predictors to be larger than the number of observations. We finally demonstrate that the sufficient forecasting improves upon the linear forecasting in both simulation studies and an empirical study of forecasting macroeconomic variables.