Overview: Machine learning is often portrayed as a value-neutral endeavor; even when that is not the exact position taken, it is implicit in how the research is carried out and how the results are communicated. This paper undertakes a qualitative analysis of the top 100 most cited papers from NeurIPS and ICML to uncover some of the most prominent values these papers espouse and how they shape the path forward. As we get a higher proliferation of AI in various aspects of our lives, critical scholars have raised concerns about the negative impacts of these systems on society. Yet, most technical papers published today pay little to no attention to the societal implications of their work. And this is despite emerging requirements like "Broader Impact Statements" that have become mandatory at several conferences. Through the manual analysis of 100 papers, this research surfaces trends that support this position and articulates that machine learning is not value-neutral.

Machine learning (ML) currently exerts an outsized influence on the world, increasingly affecting communities and institutional practices. It is therefore critical that we question vague conceptions of the field as value-neutral or universally beneficial, and investigate what specific values the field is advancing. In this paper, we present a rigorous examination of the values of the field by quantitatively and qualitatively analyzing 100 highly cited ML papers published at premier ML conferences, ICML and NeurIPS. We annotate key features of papers which reveal their values: how they justify their choice of project, which aspects they uplift, their consideration of potential negative consequences, and their institutional affiliations and funding sources. We find that societal needs are typically very loosely connected to the choice of project, if mentioned at all, and that consideration of negative consequences is extremely rare. We identify 67 values that are uplifted in machine learning research, and, of these, we find that papers most frequently justify and assess themselves based on performance, generalization, efficiency, researcher understanding, novelty, and building on previous work. We present extensive textual evidence and analysis of how these values are operationalized. Notably, we find that each of these top values is currently being defined and applied with assumptions and implications generally supporting the centralization of power. Finally, we find increasingly close ties between these highly cited papers and tech companies and elite universities.