In recent years, developing compact and efficient large language models (LLMs) has emerged as a thriving area of research. Traditional Supervised Fine-Tuning (SFT), which relies on singular ground truth labels, often fails to capture token-level dependencies and linguistic diversity. To address these limitations, we propose a logits-based fine-tuning framework that integrates the strengths of supervised learning and knowledge distillation. Our approach constructs enriched training targets by combining teacher logits with ground truth labels, preserving both correctness and linguistic diversity. This ensures more reliable and effective training. We constructed a large-scale 1.2M logits dataset and trained a series of science-focused models. Experimental results demonstrate that our method achieves significant improvements, with accuracy gains of 18% on Mawps and 22.7% on TabMWP. Across nine widely used mathematical benchmarks, our method consistently outperforms prior SFT models, achieving an average improvement of 7.28%. Codes are available at https://github.com/dvlab-research/Logits-Based-Finetuning.

Getting digitally cloned was easier than Devin Finley expected it to be. The voice-over artist, who also works as a model and bar manager, entered a studio in Manhattan last spring and read a script from a teleprompter. Across the room, a man with a large camera working for Hour One, a Tel Aviv–based video agency specializing in providing clients with lifelike virtual humans, filmed Finley from the waist up. Over Zoom, a director offered instructions about how much to move his hands. He was done in less than an hour.

Cognitive overload happens to students and teachers. Often looking like ADHD, cognitive overload can happen for a variety of reasons including challenges to your working memory. Todd Finley some ways to help your students and yourself when you struggle with cognitive overload. What is it, and how do we work with it in our students and in ourselves? Today thought leader Todd Finley is going to help us understand this. I know Todd from writing also Edutopia.