h2o-danube-1
H2O-Danube-1.8B Technical Report
Singer, Philipp, Pfeiffer, Pascal, Babakhin, Yauhen, Jeblick, Maximilian, Dhankhar, Nischay, Fodor, Gabor, Ambati, Sri Satish
We leverage and refine various techniques for pre-training large language models. Although our model is trained on significantly fewer total tokens compared to reference models of similar size, it exhibits highly competitive metrics across a multitude of benchmarks. We additionally release a chat model trained with supervised fine-tuning followed by direct preference optimization.