vaultgemma
VaultGemma: A Differentially Private Gemma Model
Sinha, Amer, Mesnard, Thomas, McKenna, Ryan, Liu, Daogao, Choquette-Choo, Christopher A., Huang, Yangsibo, Yu, Da, Kaissis, George, Charles, Zachary, Liu, Ruibo, Chua, Lynn, Kamath, Pritish, Manurangsi, Pasin, He, Steve, Zhang, Chiyuan, Ghazi, Badih, Pigem, Borja De Balle, Eruvbetine, Prem, Warkentin, Tris, Joulin, Armand, Kumar, Ravi
We introduce VaultGemma 1B, a 1 billion parameter model within the Gemma family, fully trained with differential privacy. Pretrained on the identical data mixture used for the Gemma 2 series, VaultGemma 1B represents a significant step forward in privacy-preserving large language models. We openly release this model to the community