Instruction tuning has shown its ability to not only enhance zero-shot generalization across various tasks but also its effectiveness in improving the performance of specific tasks. A crucial aspect in instruction tuning for a particular task is a strategic selection of related tasks that offer meaningful supervision, thereby enhancing efficiency and preventing performance degradation from irrelevant tasks. Our research reveals that leveraging instruction information \textit{alone} enables the identification of pertinent tasks for instruction tuning. This approach is notably simpler compared to traditional methods that necessitate complex measurements of pairwise transferability between tasks or the creation of data samples for the target task. Furthermore, by additionally learning the unique instructional template style of the meta-dataset, we observe an improvement in task selection accuracy, which contributes to enhanced overall performance. Experimental results demonstrate that training on a small set of tasks, chosen solely based on the instructions, leads to substantial performance improvements on benchmarks like P3, Big-Bench, NIV2, and Big-Bench Hard. Significantly, these improvements exceed those achieved by prior task selection methods, highlighting the efficacy of our approach.

As large language models (LLMs) see increasing adoption across the globe, it is imperative for LLMs to be representative of the linguistic diversity of the world. India is a linguistically diverse country of 1.4 Billion people. To facilitate research on multilingual LLM evaluation, we release IndicGenBench - the largest benchmark for evaluating LLMs on user-facing generation tasks across a diverse set 29 of Indic languages covering 13 scripts and 4 language families. IndicGenBench is composed of diverse generation tasks like cross-lingual summarization, machine translation, and cross-lingual question answering. IndicGenBench extends existing benchmarks to many Indic languages through human curation providing multi-way parallel evaluation data for many under-represented Indic languages for the first time. We evaluate a wide range of proprietary and open-source LLMs including GPT-3.5, GPT-4, PaLM-2, mT5, Gemma, BLOOM and LLaMA on IndicGenBench in a variety of settings. The largest PaLM-2 models performs the best on most tasks, however, there is a significant performance gap in all languages compared to English showing that further research is needed for the development of more inclusive multilingual language models. IndicGenBench is released at www.github.com/google-research-datasets/indic-gen-bench

With the information explosion on the Web, search and recommendation are foundational infrastructures to satisfying users' information needs. As the two sides of the same coin, both revolve around the same core research problem, matching queries with documents or users with items. In the recent few decades, search and recommendation have experienced synchronous technological paradigm shifts, including machine learning-based and deep learning-based paradigms. Recently, the superintelligent generative large language models have sparked a new paradigm in search and recommendation, i.e., generative search (retrieval) and recommendation, which aims to address the matching problem in a generative manner. In this paper, we provide a comprehensive survey of the emerging paradigm in information systems and summarize the developments in generative search and recommendation from a unified perspective. Rather than simply categorizing existing works, we abstract a unified framework for the generative paradigm and break down the existing works into different stages within this framework to highlight the strengths and weaknesses. And then, we distinguish generative search and recommendation with their unique challenges, identify open problems and future directions, and envision the next information-seeking paradigm.

Open-source multimodal large language models (MLLMs) excel in various tasks involving textual and visual inputs but still struggle with complex multimodal mathematical reasoning, lagging behind proprietary models like GPT-4V(ision) and Gemini-Pro. Although fine-tuning with intermediate steps (i.e., rationales) elicits some mathematical reasoning skills, the resulting models still fall short in visual comprehension due to inadequate visual-centric supervision, which leads to inaccurate interpretation of math figures. To address this issue, we propose a two-step training pipeline VCAR, which emphasizes the Visual Comprehension training in Addition to mathematical Reasoning learning. It first improves the visual comprehension ability of MLLMs through the visual description generation task, followed by another training step on generating rationales with the assistance of descriptions. Experimental results on two popular benchmarks demonstrate that VCAR substantially outperforms baseline methods solely relying on rationale supervision, especially on problems with high visual demands.

Vision-language models enable open-world classification of objects without the need for any retraining. While this zero-shot paradigm marks a significant advance, even today's best models exhibit skewed performance when objects are dissimilar from their typical depiction. Real world objects such as pears appear in a variety of forms -- from diced to whole, on a table or in a bowl -- yet standard VLM classifiers map all instances of a class to a \it{single vector based on the class label}. We argue that to represent this rich diversity within a class, zero-shot classification should move beyond a single vector. We propose a method to encode and account for diversity within a class using inferred attributes, still in the zero-shot setting without retraining. We find our method consistently outperforms standard zero-shot classification over a large suite of datasets encompassing hierarchies, diverse object states, and real-world geographic diversity, as well finer-grained datasets where intra-class diversity may be less prevalent. Importantly, our method is inherently interpretable, offering faithful explanations for each inference to facilitate model debugging and enhance transparency. We also find our method scales efficiently to a large number of attributes to account for diversity -- leading to more accurate predictions for atypical instances. Finally, we characterize a principled trade-off between overall and worst class accuracy, which can be tuned via a hyperparameter of our method. We hope this work spurs further research into the promise of zero-shot classification beyond a single class vector for capturing diversity in the world, and building transparent AI systems without compromising performance.

Cluster analysis is a popular unsupervised learning tool used in many disciplines to identify heterogeneous sub-populations within a sample. However, validating cluster analysis results and determining the number of clusters in a data set remains an outstanding problem. In this work, we present a global criterion called the Distinguishability criterion to quantify the separability of identified clusters and validate inferred cluster configurations. Our computational implementation of the Distinguishability criterion corresponds to the Bayes risk of a randomized classifier under the 0-1 loss. We propose a combined loss function-based computational framework that integrates the Distinguishability criterion with many commonly used clustering procedures, such as hierarchical clustering, k-means, and finite mixture models. We present these new algorithms as well as the results from comprehensive data analysis based on simulation studies and real data applications.

Meta's drive to integrate artificial intelligence into its products yielded strong financial results for the second quarter in a row. The company plans to spend even more on AI in the coming months, though, and its share price slumped more than 12% as the company reported earnings Wednesday. A weak sales forecast and higher spending guidance rattled investors. Revenue at the world's largest social media business increased 27% to 36.46bn during the first quarter in contrast to analyst expectations of 36.16bn. Earnings per share more than doubled to 4.71, surpassing expectations on Wall Street of 4.32.

Artificial Intelligence (AI) – models that process large and diverse datasets and make predictions from them – can have many uses in nature conservation, such as remote monitoring (like the use of camera traps to study animals or plants) or data analysis. Some of these are controversial because AI can be trained to be biased, but others are valuable research tools. Biologist Daniele Silvestro has developed an AI tool that can help identify conservation and restoration priorities. We asked him to tell us more about how it works and what it offers. Artificial intelligence (AI) is a term indicating a broad family of models used to process large and diverse datasets and make predictions from them. We built a model using biodiversity datasets as well as socioeconomic data.

Tesla reported a 55 percent drop in profit amid fierce competition in the electric vehicle market, but shares rallied on plans to accelerate the production of more affordable models. The Austin, Texas-based company on Tuesday reported profits of 1.1bn in the first quarter, down from 2.51bn a year ago. But shares of Tesla soared by 11 percent after CEO Elon Musk said that production of new, more affordable vehicles would begin in the second half of next year "if not late this year". The models "will use new aspects of the next generation platform as well as aspects of our current platform", Musk said on a conference call with analysts. Musk did not elaborate on the new vehicles, saying more details would be released in August.

The world is facing the collapse of the 1948 international order established in the wake of World War II, amid the brutal wars in Gaza and Ukraine, while authoritarian policies continue to spread, Amnesty International has warned. The report accused the world's most powerful governments, including China, Russia and the United States, of leading the global disregard for international rules and values enshrined in the Universal Declaration of Human Rights of December 1948. The war in Gaza, which began on October 7, was a "descent into hell", Secretary-General Agnes Callamard wrote in her preface to the report, where "the'never again' moral and legal lessons [of 1948] were torn into a million pieces". Noting that Hamas had committed "horrific crimes" in its assault on communities in southern Israel on October 7, Callamard said Israel's "campaign of retaliation" had become a "campaign of collective punishment". Amnesty said while Israel continued to disregard international human rights law, the US, its foremost ally, and other countries including the United Kingdom and Germany were guilty of "grotesque double standards" given their willingness to back Israeli and US authorities over Gaza while condemning war crimes by Russia in Ukraine.