Competitive Benchmarking: Lessons Learned from the Trading Agent Competition

In many real-life domains, such as trading environments, selfinterested entities need to operate subject to limited time and information. Additionally, the web has mediated an ever broader range of transactions, urging participants to concurrently trade across multiple markets. All these have generated the need for technologies that empower prompt investigation of large volumes of data and rapid evaluation of numerous alternative strategies in the face of constantly changing market conditions (Bichler, Gupta, and Ketter 2010). AI and machine-learning techniques, including neural networks and genetic algorithms, are continuously gaining ground in the support of such trading scenarios. User modeling, price forecasting, market equilibrium prediction, and strategy optimization are typical cases where AI typically provides reliable solutions. Yet, the adoption and deployment of AI practices in real trading environments remains limited, since the proprietary nature of markets precludes open benchmarking, which is critical for further scientific progress.