One of the key challenges in machine learning is to find interpretable representations of learned functions.

In the literature of studying training dynamics of transformers, several simplifications are commonly adopted such as weight reparameter-ization, attention linearization, special initialization, and lazy regime.

Large language models (LLMs) with billions of parameters demonstrate impressive performance.

Surveys have recently gained popularity as a tool to study large language models.

MMSite is a two-stage ("First Align, Then Fuse") Understanding these sites is essential for elucidating enzyme mechanisms, designing inhibitors, and developing novel drugs. MNER, high-quality datasets for multi-modal active sites identification are less abundant.

Language-image pre-training is an effective technique for learning powerful representations in general domains.

Therefore, for better cross-task generalization, the "zero-shot" learning ability of LLMs is crucial for

Trained on vast corpora of human language, language models demonstrate emergent human-like reasoning abilities.