In this technical report, we present Zamba, a novel 7B SSM-transformer hybrid model which achieves competitive performance against leading open-weight models at a comparable scale. Zamba is trained on 1T tokens from openly available datasets and is the best non-transformer model at this scale. Zamba pioneers a unique architecture combining a Mamba backbone with a single shared attention module, thus obtaining the benefits of attention at minimal parameter cost. Due to its architecture, Zamba is significantly faster at inference than comparable transformer models and requires substantially less memory for generation of long sequences. Zamba is pretrained in two phases: the first phase is based on existing web datasets, while the second one consists of annealing the model over high-quality instruct and synthetic datasets, and is characterized by a rapid learning rate decay. We open-source the weights and all checkpoints for Zamba, through both phase 1 and annealing phases.

If you've looked at videos collected from trail cameras, you might have found that a large fraction of them contain no visible animals. And if you've spent much time looking at blank videos, you might wish there was a better way! Using automatic classification from Zamba, an AI tool for wildlife research and conservation, you can eliminate a substantial fraction of blank videos, sight unseen, while losing only a small fraction of videos that actually contain animals. The goal of this article is to quantify that claim. Here's how we'll do it: To train Zamba's classification model, we collected more than 280,000 videos from researchers at the Max Planck Institute for Evolutionary Anthropology working in West, Central, and East Africa.

Pfizer and Saama Technologies will work together to develop and deploy an artificial intelligence (AI) powered analytical tool geared toward clearing many of the obstacles faced by study data managers and monitors. The Saama Life Science Analytics Cloud (LSAC) platform will aggregate, analyze, model and predict data via'deep learning'. Pfizer's role is to provide the required clinical data and domain knowledge to train Saama models to obtain the accuracy needed. Demetris Zambas, VP and head of data monitoring and management at Pfizer, told Outsourcing-Pharma that Saama was invited to work with his company on the AI project after a'hackathon', in which a number of companies competed with each other. Saama initially was invited to participate based on their core competencies.