Our evaluation and analysis are conducted mainly on the group of models listed in Table 2 in the13 main paper. We have chosen models such that they cover most of the popular and best-performing14 methods used by recent multimodal understanding work. In this part, we discuss all the models we15 have used in our experiments and explain their evaluation details, the public checkpoints we have16 chosen, and display the prompts we used to adapt the model to our datasets.17 During evaluation, we chose to follow the standard prompt provided by the authors whenever possi-18 ble for multiple-choice and short-answer questions. When the prompt is not provided for the model,19 we select a custom prompt that is created through several iterations of prompt engineering to select20 the one that produces the most effective results. The images are always included as the prefix.21 We used three proprietary models in our evaluation: GPT-o4-mini [1], Gem-22 ini 1.5 Pro [9], and Claude 3 Haiku [10]. Below we note the model API version used for evaluation.23 GPT-o4-mini: May 13-15, 2025.24 Cambrian-1 is a recent state-of-the-art model that excels at visual-centric tasks.27 This model explores combinations of vision encoders, text and image integration techniques, and28 instruction tuning strategies. We use the official implementation and checkpoint1 with a LLaMA3-29 8B-Instruct LLM backbone model in our evaluation.30 InternVL scales up the vision foundation model while aligning it with the back-31 bone LLM, and is trained on web-scale image-text data to achieve strong performance across a vari-32 ety of vision-centric tasks. We use the official implementation and checkpoint2 with the InternViT-33 300M-448px vision backbone and Internlm2.5-7B-chat LLaMA-3.2 is the first collection of multimodal large language model from the35 LLaMA family that was previously text-only. The integration of vision involves utilizing cross-36 attention layers and a pre-trained vision encoder that feeds directly into the text-processor. The37 model follows a commonly used training recipe that includes pretraining on noisy image-text pairs38 and then high-quality knowledge enhanced pairs. Notably, the language-model parameters were39 frozen during the training of alignment of image and text to retain strong text-only capabilities. We40 use the official implementation and checkpoint3 that uses a LLaMA-3.1 text-only language backbone41 in our evaluation. When evaluating the model, we choose to use a custom prompt since no standard42 prompt is provided.43

Our findings reveal that: I)LLMs with higher accuracy may exhibit lower certainty; II) Larger-scale LLMs may display greater uncertainty compared to their smaller counterparts; and III)Instruction-finetuning tends to increase the uncertainty of LLMs. These results underscore the significance of incorporating uncertainty into the evaluation of LLMs.

Artificial intelligence agents are increasingly capable of performing complex tasks across various domains. As these agents advance, there is a growing need to accurately measure and benchmark their capabilities, particularly in accelerating AI research and development. Current benchmarks focus on general machine learning tasks, but lack comprehensive evaluation methods for assessing AI agents' abilities in tackling research-level problems and competition-level challenges in the field of AI. We present the ML Research Benchmark (MLRB), comprising 7 competition-level tasks derived from recent machine learning conference tracks. These tasks span activities typically undertaken by AI researchers, including model training efficiency, pretraining on limited data, domain specific fine-tuning, and model compression. This paper introduces a novel benchmark and evaluates it using agent scaffolds powered by frontier models, including Claude-3 and GPT-4o. The results indicate that the Claude-3.5 Sonnet agent performs best across our benchmark, excelling in planning and developing machine learning models. However, both tested agents struggled to perform non-trivial research iterations. We observed significant performance variations across tasks, highlighting the complexity of AI development and the challenges in creating versatile agent scaffolds. While current AI agents can successfully navigate complex instructions and produce baseline results, they fall short of the capabilities required for advanced AI research. The ML Research Benchmark provides a valuable framework for assessing and comparing AI agents on tasks mirroring real-world AI research challenges.

Assistive robots have attracted significant attention due to their potential to enhance the quality of life for vulnerable individuals like the elderly. The convergence of computer vision, large language models, and robotics has introduced the `visuolinguomotor' mode for assistive robots, where visuals and linguistics are incorporated into assistive robots to enable proactive and interactive assistance. This raises the question: \textit{In circumstances where visuals become unreliable or unavailable, can we rely solely on language to control robots, i.e., the viability of the `linguomotor` mode for assistive robots?} This work takes the initial steps to answer this question by: 1) evaluating the responses of assistive robots to language prompts of varying granularities; and 2) exploring the necessity and feasibility of controlling the robot on-the-fly. We have designed and conducted experiments on a Sawyer cobot to support our arguments. A Turtlebot robot case is designed to demonstrate the adaptation of the solution to scenarios where assistive robots need to maneuver to assist. Codes will be released on GitHub soon to benefit the community.

Tokenization, the division of input text into input tokens, is an often overlooked aspect of the large language model (LLM) pipeline and could be the source of useful or harmful inductive biases. Historically, LLMs have relied on byte pair encoding, without care to specific input domains. With the increased use of LLMs for reasoning, various number-specific tokenization schemes have been adopted, with popular models like LLaMa and PaLM opting for single-digit tokenization while GPT-3.5 and GPT-4 have separate tokens for each 1-, 2-, and 3-digit numbers. In this work, we study the effect this choice has on numerical reasoning through the use of arithmetic tasks. We consider left-to-right and right-to-left tokenization for GPT-3.5 and -4, finding that right-to-left tokenization (enforced by comma separating numbers at inference time) leads to largely improved performance. Furthermore, we find that model errors when using standard left-to-right tokenization follow stereotyped error patterns, suggesting that model computations are systematic rather than approximate. We show that the model is able to convert between tokenizations easily, thus allowing chain-of-thought-inspired approaches to recover performance on left-to-right tokenized inputs. We also find the gap between tokenization directions decreases when models are scaled, possibly indicating that larger models are better able to override this tokenization-dependent inductive bias. In summary, our work performs the first study of how number tokenization choices lead to differences in model performance on arithmetic tasks, accompanied by a thorough analysis of error patterns. We hope this work inspires practitioners to more carefully ablate number tokenization-related choices when working towards general models of numerical reasoning.

We investigate the predictability of large language model (LLM) capabilities: given records of past experiments using different model families, numbers of parameters, tasks, and numbers of in-context examples, can we accurately predict LLM performance on new experiment configurations? Answering this question has practical implications for LLM users (e.g., deciding which models to try), developers (e.g., prioritizing evaluation on representative tasks), and the research community (e.g., identifying hard-to-predict capabilities that warrant further investigation). We study the performance prediction problem on experiment records from BIG-bench. On a random train-test split, an MLP-based predictor achieves an $R^2$ score greater than 95%, indicating the presence of learnable patterns within the experiment records. We then formulate the problem of searching for "small-bench," an informative subset of BIG-bench tasks from which the performance on the full set can be maximally recovered. We find a subset as informative as BIG-bench Hard for evaluating new model families, while being $3\times$ smaller. Additionally, we find competitive subsets by clustering task representations learned by our MLP-based predictor and selecting tasks close to cluster centroids, highlighting the importance of task diversity in constructing "small-bench."

Get the Spectroscopy and Chemometrics News Weekly in real time on Twitter @ CalibModel and follow us. "Foods: Prediction Models for the Content of Calcium, Boron and Potassium in the Fruit of'Huangguan' Pears Established by Using Near-Infrared Spectroscopy" LINK "Construction and Application of Detection Model for Leucine and Tyrosine Content in Golden Tartary Buckwheat Based on Near Infrared Spectroscopy" LINK "Rapid recognition of different sources of methamphetamine drugs based on hand-held near infrared spectroscopy and multi-layer-extreme learning machine algorithms" LINK "Rapid determination of viscosity and viscosity index of lube base oil based on near-infrared spectroscopy and new transformation formula" LINK "Simple dilated convolutional neural network for quantitative modeling based on near infrared spectroscopy techniques" LINK "Fast and nondestructive discrimination of fresh tea leaves at different altitudes based on near infrared spectroscopy and various chemometrics methods" LINK "NIR spectroscopy combined with 1D-convolutional neural network for breast cancerization analysis and diagnosis" LINK "Associations between visceral adipose tissue estimates produced by near-infrared spectroscopy, mobile anthropometrics, and traditional body composition …" LINK "Discrimination of Minced Mutton Adulteration Based on Sized-Adaptive Online NIRS Information and 2D Conventional Neural Network. "Fruit detection research based on near-infrared spectroscopy and lightweight neural network" LINK "Honey quality detection based on near-infrared spectroscopy" LINK "Evaluation of the potential of near infrared hyperspectral imaging for monitoring the invasive brown marmorated stink bug" LINK "Denoising stacked autoencodersbased nearinfrared quality monitoring method via robust samples evaluation" LINK "Visualization research of egg freshness based on hyperspectral imaging and binary competitive adaptive reweighted sampling" LINK "Desert Soil Salinity Inversion Models Based on Field In Situ Spectroscopy in Southern Xinjiang, China" LINK "Novel broad spectral response perovskite solar cells: A review of the current status and advanced strategies for breaking the theoretical limit efficiency" LINK "Remote Sensing: Estimation of Potato Above-Ground Biomass Based on Vegetation Indices and Green-Edge Parameters Obtained from UAVs" LINK "Prognostic value of syntax score, intravascular ultrasound and near-infrared spectroscopy to identify low-risk patients with coronary artery disease 5-year …" LINK

It seems like we're on the cusp of doing amazing things with chatbots and data mining, which can augment manual information architecture work and result in a better user experience overall. Automated information processing can help us identify search patterns and recommend information structures, to improve the findability of content. Let's say you have a service that includes a search component. Right now, your users might be running searches and using manual filters to sort through the results, as search users are wont to do. How do you know if they are finding what they need?

Google announced today that its Cloud Video Intelligence API is generally available, along with a new Content Classification feature for its Cloud Natural Language API. The updates are aimed at giving customers new capabilities for making their applications smarter using prebuilt machine learning systems. As their names imply, the two services are designed to provide developers with tools that they can use to make applications understand the content of videos and text. That, in turn, is supposed to help with the creation of more intelligent apps that would have previously been the domain of the tech titans. In addition to becoming generally available, the Cloud Video Intelligence API can now be used to transcribe the contents of video fed into it, along with its existing support for detecting objects within footage, spotting shot changes in a single video, and spotting explicit content in footage.

Adjusted R² measures the goodness of fit of a regression model. Higher the R², better is the model.