Towards Reinforcement Learning for Exploration of Speculative Execution Vulnerabilities

--Speculative execution attacks such as Spectre can be used to bypass the security isolation and steal information from other programs. Exploring speculative execution attacks on existing processors requires intensive manual reverse engineering and intimate knowledge of the processor . This reverse engineering-based approach requires extensive human effort, which is slow and not scalable. In this paper, we introduce SpecRL, a framework that utilizes reinforcement learning to explore speculative execution leaks in commercial-of-the shelf microprocessors. This reinforcement learning agent approach requires less reverse engineering effort while still be able to identify speculative execution vulnerabilties.