Peer assessment of CS doctoral programs shows strong correlation with faculty citations

Rankings of universities and specialized academic programs have a major influence on students deciding what university to attend, faculty deciding where to work, government bodies deciding where and how to invest education and research funding, and university leaders deciding how to grow their institutions.9 There is general agreement in scientometrics that the quality of a university or a program depends on many factors, and different ranking metrics might be appropriate for different types of users. However, major points of contention emerge when it comes to agreeing on ranking methodology.20 Given the increasing impact of rankings, there is a need to better understand the actors influencing rankings and come up with a justifiable, transparent formula that encourages high-quality education and research at universities.11 We aim to contribute toward achieving this objective by focusing on ranking of the U.S. doctoral programs in computer science. We broadly group quality measures into objective (such as average research funding per faculty member) and subjective (such as peer assessment). The influential U.S. News ranking of computer science doctoral programsa is based purely on peer assessment in which computer science department chairs are asked to score other computer science programs on a scale of 1 to 5, with 1 being "marginal" and 5 being "outstanding," or enter "do not know" if not sufficiently familiar with the program. The final ranking is obtained by averaging the individual scores.