Recent work in machine learning increasingly attributes human-like capabilities such as reasoning or theory of mind to large language models (LLMs) on the basis of benchmark performance. This paper examines this practice through the lens of construct validity, understood as the problem of linking theoretical capabilities to their empirical measurements. It contrasts three influential frameworks: the nomological account developed by Cronbach and Meehl, the inferential account proposed by Messick and refined by Kane, and Borsboom's causal account. I argue that the nomological account provides the most suitable foundation for current LLM capability research. It avoids the strong ontological commitments of the causal account while offering a more substantive framework for articulating construct meaning than the inferential account. I explore the conceptual implications of adopting the nomological account for LLM research through a concrete case: the assessment of reasoning capabilities in LLMs.

In each competition, numerous practitioners repeatedly evaluated their progress against a holdout set that forms the basis of a public ranking availablethroughout the competition. Performance on a separate test set used only oncedetermined the final ranking.

Using these smoothed rank andsort operators, we propose differentiable proxies for the classification 0/1 loss as well as for the quantileregressionloss.

Moreover, given the ever increasing number of machine learning models being developed, model selection is becoming increasingly important.

Building on this result, we show several examples of how variance of predictions can be exploited in learning.

Thismakesinferenceveryfast,andmaynot even need to use all input featuresto make a prediction (with axis-aligned trees).

Deep learning (DL) algorithms rely on massive amounts of labeled data.

Ournew bound consistently beats state-of-the-art bounds both on a toy example and on UCI datasets (with large enoughn).

Thestandard supervised learning paradigm workseffectivelywhentraining data shares the same distribution as the upcoming testing samples.

However,in manyapplications there are several different parameters one might wish to vary: for example, scale and density. In contrast to the one-parameter setting, techniques for applying statistics and machine learning in the setting of multiparameter persistence are not well understood due to the lack of a concise representationoftheresults.