Deep Reinforcement Learning for De-Novo Drug Design

We propose a novel computational strategy based on deep and reinforcement learning techniques for de-novo design of molecules with desired properties. This strategy integrates two deep neural networks - generative and predictive - that are trained separately but employed jointly to generate novel chemical structures with the desired properties. Generative models are trained to produce chemically feasible SMILES, and predictive models are derived to forecast the desired compound properties. One example of such an approach is the broad use of Lipinski's rules of bioavailability (15, 16) to filter molecules that possess the desired bioactivity in vitro. Indeed, it has been acknowledged that the broad use of these rules has substantially reduced the failure rate in experimental ADME studies of drug candidates (17). The crucial step in many new drug discovery projects is the formulation of a well-motivated hypothesis for new lead compound generation (de novo design) or compound selection from available or synthetically feasible chemical libraries based on the available SAR data. Commonly, an interdisciplinary team of scientists generates the new hypothesis by employing computational models of drug action and relying on their expertise and medicinal chemistry intuition. Therefore, the design hypothesis is often biased towards preferred chemistry (18) or driven by model interpretation (19). Automated approaches for designing compounds with desired properties de novo have become an active field of research in the last 15 years (20, 21). In an attempt to design new compounds, both medicinal and computational chemists face virtually infinite chemical space. Great advances in both computational algorithms(24, 25), hardware, and high-throughput screening (HTS) technologies (16) notwithstanding, the size of this virtual library prohibits its exhaustive sampling and testing by systematic construction and evaluation of each individual compound. Local optimization approaches have been proposed but they do not ensure the optimal solution, as the design process converges on a local or'practical' optimum by stochastic sampling, or restrict the search to a defined section of chemical space which can be screened exhaustively (20, 26-28).