MatterChat: A Multi-Modal LLM for Material Science

In-silico material discovery and design have traditionally relied on high-fidelity first-principles methods such as density functional theory (DFT) [1] and ab-initio molecular dynamics (AIMD) [2] to accurately model atomic interactions and predict material properties. Despite their effectiveness, these methods face significant challenges due to their prohibitive computational cost, limiting their scalability for highthroughput screening across vast chemical spaces and for simulations over large length and time scales. Moreover, many advanced materials remain beyond the reach of widespread predictive theories due to a fundamental lack of mechanistic understanding. These challenges stem from the inherent complexity of their chemical composition, phase stability, and the intricate interplay of multiple order parameters, compounded by the lack of self-consistent integration between theoretical models and multi-modal experimental findings. As a result, breakthroughs in functional materials, such as new classes of correlated oxides, nitrides, and low-dimensional quantum materials, have largely been serendipitous or guided by phenomenological intuition rather than systematic, theory-driven design. Attempts to predict new materials and functionalities have often led to mixed results, with theoretically proposed systems failing to exhibit the desired properties when synthesized and tested.