Weissenberger, Felix
Towards an AI co-scientist
Gottweis, Juraj, Weng, Wei-Hung, Daryin, Alexander, Tu, Tao, Palepu, Anil, Sirkovic, Petar, Myaskovsky, Artiom, Weissenberger, Felix, Rong, Keran, Tanno, Ryutaro, Saab, Khaled, Popovici, Dan, Blum, Jacob, Zhang, Fan, Chou, Katherine, Hassidim, Avinatan, Gokturk, Burak, Vahdat, Amin, Kohli, Pushmeet, Matias, Yossi, Carroll, Andrew, Kulkarni, Kavita, Tomasev, Nenad, Guan, Yuan, Dhillon, Vikram, Vaishnav, Eeshit Dhaval, Lee, Byron, Costa, Tiago R D, Penadés, José R, Peltz, Gary, Xu, Yunhan, Pawlosky, Annalisa, Karthikesalingam, Alan, Natarajan, Vivek
Scientific discovery relies on scientists generating novel hypotheses that undergo rigorous experimental validation. To augment this process, we introduce an AI co-scientist, a multi-agent system built on Gemini 2.0. The AI co-scientist is intended to help uncover new, original knowledge and to formulate demonstrably novel research hypotheses and proposals, building upon prior evidence and aligned to scientist-provided research objectives and guidance. The system's design incorporates a generate, debate, and evolve approach to hypothesis generation, inspired by the scientific method and accelerated by scaling test-time compute. Key contributions include: (1) a multi-agent architecture with an asynchronous task execution framework for flexible compute scaling; (2) a tournament evolution process for self-improving hypotheses generation. Automated evaluations show continued benefits of test-time compute, improving hypothesis quality. While general purpose, we focus development and validation in three biomedical areas: drug repurposing, novel target discovery, and explaining mechanisms of bacterial evolution and anti-microbial resistance. For drug repurposing, the system proposes candidates with promising validation findings, including candidates for acute myeloid leukemia that show tumor inhibition in vitro at clinically applicable concentrations. For novel target discovery, the AI co-scientist proposed new epigenetic targets for liver fibrosis, validated by anti-fibrotic activity and liver cell regeneration in human hepatic organoids. Finally, the AI co-scientist recapitulated unpublished experimental results via a parallel in silico discovery of a novel gene transfer mechanism in bacterial evolution. These results, detailed in separate, co-timed reports, demonstrate the potential to augment biomedical and scientific discovery and usher an era of AI empowered scientists.
Decoupling Hierarchical Recurrent Neural Networks With Locally Computable Losses
Mujika, Asier, Weissenberger, Felix, Steger, Angelika
Learning long-term dependencies is a key long-standing challenge of recurrent neural networks (RNNs). Hierarchical recurrent neural networks (HRNNs) have been considered a promising approach as long-term dependencies are resolved through shortcuts up and down the hierarchy. Yet, the memory requirements of Truncated Backpropagation Through Time (TBPTT) still prevent training them on very long sequences. In this paper, we empirically show that in (deep) HRNNs, propagating gradients back from higher to lower levels can be replaced by locally computable losses, without harming the learning capability of the network, over a wide range of tasks. This decoupling by local losses reduces the memory requirements of training by a factor exponential in the depth of the hierarchy in comparison to standard TBPTT.