We present initial results of a forthcoming benchmark for evaluating LLM agents on white-collar tasks of economic value. We evaluate agents on real-world "messy" open-web research tasks of the type that are routine in finance and consulting. In doing so, we lay the groundwork for an LLM agent evaluation suite where good performance directly corresponds to a large economic and societal impact. We built and tested several agent architectures with o1-preview, GPT-4o, Claude-3.5 Sonnet, Llama 3.1 (405b), and GPT-4o-mini. On average, LLM agents powered by Claude-3.5 Sonnet and o1-preview substantially outperformed agents using GPT-4o, with agents based on Llama 3.1 (405b) and GPT-4o-mini lagging noticeably behind. Across LLMs, a ReAct architecture with the ability to delegate subtasks to subagents performed best. In addition to quantitative evaluations, we qualitatively assessed the performance of the LLM agents by inspecting their traces and reflecting on their observations. Our evaluation represents the first in-depth assessment of agents' abilities to conduct challenging, economically valuable analyst-style research on the real open web.

Work is fundamental to societal prosperity and mental health, providing financial security, identity, purpose, and social integration. The emergence of generative artificial intelligence (AI) has catalysed debate on job displacement. Some argue that many new jobs and industries will emerge to offset the displacement, while others foresee a widespread decoupling of economic productivity from human input threatening jobs on an unprecedented scale. This study explores the conditions under which both may be true and examines the potential for a self-reinforcing cycle of recessionary pressures that would necessitate sustained government intervention to maintain job security and economic stability. A system dynamics model was developed to undertake ex ante analysis of the effect of AI-capital deepening on labour underutilisation and demand in the economy. Results indicate that even a moderate increase in the AI-capital-to-labour ratio could increase labour underutilisation to double its current level, decrease per capita disposable income by 26% (95% interval, 20.6% - 31.8%), and decrease the consumption index by 21% (95% interval, 13.6% - 28.3%) by mid-2050. To prevent a reduction in per capita disposable income due to the estimated increase in underutilization, at least a 10.8-fold increase in the new job creation rate would be necessary. Results demonstrate the feasibility of an AI-capital- to-labour ratio threshold beyond which even high rates of new job creation cannot prevent declines in consumption. The precise threshold will vary across economies, emphasizing the urgent need for empirical research tailored to specific contexts. This study underscores the need for governments, civic organisations, and business to work together to ensure a smooth transition to an AI- dominated economy to safeguard the Mental Wealth of nations.

Every day, global trends are reshaping society and the business landscape. Today's infographic from McKinsey Global Institute (MGI) presents a snapshot of 10 insights into how the world is changing, based on its research work from 2019. How did we get here, and where are we going? Globalization is making the world "shrink" every day, as humans and trade become increasingly connected. However, there are signs that point to a new phase of globalization that is leading to different outcomes than prior years.

AI (artificial intelligence) and robots threaten our living standards. With the ever-increasing ability of robots and AI to perform more intricate tasks with greater accuracy, speed and consistency than humans, it is progressively becoming obvious that many jobs, if done by robots, would be more cost-effective for companies, and may even do the job better. Not only do robots and AI hold the potential to take over many repetitive manufacturing jobs, they also look likely to take over commercial jobs, such as shopping assistants, as well as higher skilled jobs: from secretaries and administrators to surgeons, with some arguing that robots with AI may be able to write their own code as part of self-improvement in the search for performing ever more complex tasks even more productively and effectively – even removing the need for computer programmers! This presents an interesting and potentially frightening predicament: a society where business owners reap the vast benefits of having substituted workers for much more productive robots, while many millions of workers are out of their jobs. There could be millions of redundant workers suddenly forced to find alternative employment, but with robots taking many jobs, the majority will struggle to retrain to find jobs well-suited to their qualifications and experience that are not being done by robots (hence their redundancy).