Overview
A Review of Reinforcement Learning in Financial Applications
Bai, Yahui, Gao, Yuhe, Wan, Runzhe, Zhang, Sheng, Song, Rui
A financial market is a marketplace where financial instruments such as stocks and bonds are bought and sold (Fama 1970). Individuals and organizations can play crucial roles in financial markets to facilitate the allocation of capital. Market participants face diverse challenges, such as portfolio management, which aims to maximize investment returns over time, and market-making, which seeks to profit from the bid-ask spread while managing inventory risk. As the volume of financial data has increased dramatically over time, new opportunities and challenges have arisen in the analysis process, leading to the increased adoption of advanced Machine Learning (ML) models. Reinforcement Learning (RL)(Sutton & Barto 2018), as one of the main categories of ML, has revolutionized the field of artificial intelligence by empowering agents to interact with the environment and allowing them to learn and improve their performance. The success of RL has been demonstrated in various fields, including games, robots, mobile health (Nash Jr 1950, Kalman 1960, Murphy 2003), etc. In finance, applications such as market making, portfolio management, and order execution can benefit from the ability of RL algorithms to learn and adapt to changing environments. Compared to traditional models that rely on statistical techniques and econometric methods such as time series models (ARMA, ARIMA), factor models, and panel models, the RL framework empowers agents to learn decision-making by interacting with an environment and deducing the consequences of past actions to maximize cumulative rewards (Charpentier et al. 2021).
GPT for Games: An Updated Scoping Review (2020-2024)
Yang, Daijin, Kleinman, Erica, Harteveld, Casper
Due to GPT's impressive generative capabilities, its applications in games are expanding rapidly. To offer researchers a comprehensive understanding of the current applications and identify both emerging trends and unexplored areas, this paper introduces an updated scoping review of 131 articles, 76 of which were published in 2024, to explore GPT's potential for games. By coding and synthesizing the papers, we identify five prominent applications of GPT in current game research: procedural content generation, mixed-initiative game design, mixed-initiative gameplay, playing games, and game user research. Drawing on insights from these application areas and emerging research, we propose future studies should focus on expanding the technical boundaries of the GPT models and exploring the complex interaction dynamics between them and users. This review aims to illustrate the state of the art in innovative GPT applications in games, offering a foundation to enrich game development and enhance player experiences through cutting-edge AI innovations.
AI for ERW Detection in Clearance Operations -- The State of Research
Kischelewski, Björn, Cathcart, Gregory, Wahl, David, Guedj, Benjamin
The clearance of explosive remnants of war (ERW) continues to be a predominantly manual and high-risk process that can benefit from advances in technology to improve its efficiency and effectiveness. In particular, research on artificial intelligence for ERW clearance has grown significantly in recent years. However, this research spans a wide range of fields, making it difficult to gain a comprehensive understanding of current trends and developments. Therefore, this article provides a literature review of academic research on AI for ERW detection for clearance operations. It finds that research can be grouped into two main streams, AI for ERW object detection and AI for ERW risk prediction, with the latter being much less studied than the former. From the analysis of the eligible literature, we develop three opportunities for future research, including a call for renewed efforts in the use of AI for ERW risk prediction, the combination of different AI systems and data sources, and novel approaches to improve ERW risk prediction performance, such as pattern-based prediction. Finally, we provide a perspective on the future of AI for ERW clearance. We emphasize the role of traditional machine learning for this task, the need to dynamically incorporate expert knowledge into the models, and the importance of effectively integrating AI systems with real-world operations.
LLM-Inference-Bench: Inference Benchmarking of Large Language Models on AI Accelerators
Chitty-Venkata, Krishna Teja, Raskar, Siddhisanket, Kale, Bharat, Ferdaus, Farah, Tanikanti, Aditya, Raffenetti, Ken, Taylor, Valerie, Emani, Murali, Vishwanath, Venkatram
Large Language Models (LLMs) have propelled groundbreaking advancements across several domains and are commonly used for text generation applications. However, the computational demands of these complex models pose significant challenges, requiring efficient hardware acceleration. Benchmarking the performance of LLMs across diverse hardware platforms is crucial to understanding their scalability and throughput characteristics. We introduce LLM-Inference-Bench, a comprehensive benchmarking suite to evaluate the hardware inference performance of LLMs. We thoroughly analyze diverse hardware platforms, including GPUs from Nvidia and AMD and specialized AI accelerators, Intel Habana and SambaNova. Our evaluation includes several LLM inference frameworks and models from LLaMA, Mistral, and Qwen families with 7B and 70B parameters. Our benchmarking results reveal the strengths and limitations of various models, hardware platforms, and inference frameworks. We provide an interactive dashboard to help identify configurations for optimal performance for a given hardware platform.
Autonomous Driving in Unstructured Environments: How Far Have We Come?
Min, Chen, Si, Shubin, Wang, Xu, Xue, Hanzhang, Jiang, Weizhong, Liu, Yang, Wang, Juan, Zhu, Qingtian, Zhu, Qi, Luo, Lun, Kong, Fanjie, Miao, Jinyu, Cai, Xudong, An, Shuai, Li, Wei, Mei, Jilin, Sun, Tong, Zhai, Heng, Liu, Qifeng, Zhao, Fangzhou, Chen, Liang, Wang, Shuai, Shang, Erke, Shang, Linzhi, Zhao, Kunlong, Li, Fuyang, Fu, Hao, Jin, Lei, Zhao, Jian, Mao, Fangyuan, Xiao, Zhipeng, Li, Chengyang, Dai, Bin, Zhao, Dawei, Xiao, Liang, Nie, Yiming, Hu, Yu, Li, Xuelong
Research on autonomous driving in unstructured outdoor environments is less advanced than in structured urban settings due to challenges like environmental diversities and scene complexity. These environments-such as rural areas and rugged terrains-pose unique obstacles that are not common in structured urban areas. Despite these difficulties, autonomous driving in unstructured outdoor environments is crucial for applications in agriculture, mining, and military operations. Our survey reviews over 250 papers for autonomous driving in unstructured outdoor environments, covering offline mapping, pose estimation, environmental perception, path planning, end-to-end autonomous driving, datasets, and relevant challenges. We also discuss emerging trends and future research directions. This review aims to consolidate knowledge and encourage further research for autonomous driving in unstructured environments. To support ongoing work, we maintain an active repository with up-to-date literature and open-source projects at: https://github.com/chaytonmin/Survey-Autonomous-Driving-in-Unstructured-Environments.
What Makes An Expert? Reviewing How ML Researchers Define "Expert"
Human experts are often engaged in the development of machine learning systems to collect and validate data, consult on algorithm development, and evaluate system performance. At the same time, who counts as an 'expert' and what constitutes 'expertise' is not always explicitly defined. In this work, we review 112 academic publications that explicitly reference 'expert' and 'expertise' and that describe the development of machine learning (ML) systems to survey how expertise is characterized and the role experts play. We find that expertise is often undefined and forms of knowledge outside of formal education and professional certification are rarely sought, which has implications for the kinds of knowledge that are recognized and legitimized in ML development. Moreover, we find that expert knowledge tends to be utilized in ways focused on mining textbook knowledge, such as through data annotation. We discuss the ways experts are engaged in ML development in relation to deskilling, the social construction of expertise, and implications for responsible AI development. We point to a need for reflection and specificity in justifications of domain expert engagement, both as a matter of documentation and reproducibility, as well as a matter of broadening the range of recognized expertise.
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies
Wang, Luping, Chen, Sheng, Jiang, Linnan, Pan, Shu, Cai, Runze, Yang, Sen, Yang, Fei
The large models, as predicted by scaling raw forecasts, have made groundbreaking progress in many fields, particularly in natural language generation tasks, where they have approached or even surpassed human levels. However, the unprecedented scale of their parameters brings significant computational and storage costs. These large models require substantial computational resources and GPU memory to operate. When adapting large models to specific downstream tasks, their massive parameter scale poses a significant challenge in fine-tuning on hardware platforms with limited computational power and GPU memory. To address this issue, Parameter-Efficient Fine-Tuning (PEFT) offers a practical solution by efficiently adjusting the parameters of large pre-trained models to suit various downstream tasks. Specifically, PEFT adjusts the parameters of pre-trained large models to adapt to specific tasks or domains, minimizing the introduction of additional parameters and the computational resources required. This review mainly introduces the preliminary knowledge of PEFT, the core ideas and principles of various PEFT algorithms, the applications of PEFT, and potential future research directions. By reading this review, we believe that interested parties can quickly grasp the PEFT methodology, thereby accelerating its development and innovation.
P-Masking: Power Law Masking Improves Multi-attribute Controlled Generation
We introduce LingGen, a novel approach for controlled text generation that offers precise control over a wide array of linguistic attributes, even as the number of attributes varies. LingGen employs a dynamic P-MASKING strategy, which samples masking rates from a power law distribution during training. This innovative approach enables the model to develop robust representations and adapt its attribute control capabilities across a variable number of attributes, from a single attribute to multiple complex configurations. The P-MASKING technique enhances LingGen's ability to manage different levels of attribute visibility, resulting in superior performance in multi-attribute generation tasks. Our experiments demonstrate that LingGen surpasses current state-of-the-art models in both attribute control accuracy and text fluency, particularly excelling in scenarios with varying attribute demands. Additionally, our ablation studies highlight the effectiveness of P-MASKING and the influence of different base language models on performance. These findings demonstrate LingGen's potential for applications requiring precise and adaptable control over multiple linguistic attributes in text generation.
Localization, balance and affinity: a stronger multifaceted collaborative salient object detector in remote sensing images
Xie, Yakun, Liu, Suning, Chen, Hongyu, Cao, Shaohan, Zhang, Huixin, Feng, Dejun, Wan, Qian, Zhu, Jun, Zhu, Qing
Despite significant advancements in salient object detection(SOD) in optical remote sensing images(ORSI), challenges persist due to the intricate edge structures of ORSIs and the complexity of their contextual relationships. Current deep learning approaches encounter difficulties in accurately identifying boundary features and lack efficiency in collaboratively modeling the foreground and background by leveraging contextual features. To address these challenges, we propose a stronger multifaceted collaborative salient object detector in ORSIs, termed LBA-MCNet, which incorporates aspects of localization, balance, and affinity. The network focuses on accurately locating targets, balancing detailed features, and modeling image-level global context information. Specifically, we design the Edge Feature Adaptive Balancing and Adjusting(EFABA) module for precise edge localization, using edge features to guide attention to boundaries and preserve spatial details. Moreover, we design the Global Distributed Affinity Learning(GDAL) module to model global context. It captures global context by generating an affinity map from the encoders final layer, ensuring effective modeling of global patterns. Additionally, deep supervision during deconvolution further enhances feature representation. Finally, we compared with 28 state of the art approaches on three publicly available datasets. The results clearly demonstrate the superiority of our method.
Deep Learning Frameworks for Cognitive Radio Networks: Review and Open Research Challenges
Jagatheesaperumal, Senthil Kumar, Ahmad, Ijaz, Höyhtyä, Marko, Khan, Suleman, Gurtov, Andrei
Deep learning has been proven to be a powerful tool for addressing the most significant issues in cognitive radio networks, such as spectrum sensing, spectrum sharing, resource allocation, and security attacks. The utilization of deep learning techniques in cognitive radio networks can significantly enhance the network's capability to adapt to changing environments and improve the overall system's efficiency and reliability. As the demand for higher data rates and connectivity increases, B5G/6G wireless networks are expected to enable new services and applications significantly. Therefore, the significance of deep learning in addressing cognitive radio network challenges cannot be overstated. This review article provides valuable insights into potential solutions that can serve as a foundation for the development of future B5G/6G services. By leveraging the power of deep learning, cognitive radio networks can pave the way for the next generation of wireless networks capable of meeting the ever-increasing demands for higher data rates, improved reliability, and security.