There are already over 50 different 1B+ parametersLLMs accessible via open-source checkpoints or proprietary APIs. That’s not counting any private models or models with academic papers but no available API or model weights. There’s even more if you count fine-tuned models like Alpaca or InstructGPT. A list of the ones I know about (this is an evolving document). GPT-J (6B) (EleutherAI) GPT-Neo (1.3B, 2.7B, 20B) (EleutherAI) Pythia (1B, 1.4B, 2.8B, 6.9B, 12B) Polyglot (1.3B, 3.8B, 5.8B) J1 (

AI21 Labs, an Israeli generative AI company, today announced its latest large language model (LLM), called Jurassic-2. Up till now, the base model of AI21 Labs has been Jurassic-1, the largest version of which has 178 billion parameters. That made it among the largest LLMs on the market -- slightly bigger than OpenAI's 175B parameter GPT-3 davinci model. However, when I spoke this week to AI21 Labs co-founder and co-CEO, Ori Goshen, he was reluctant to tell me how large Jurassic-2 is. LLM size "plays a factor, but it's not the only factor," said Goshen. "So we've stopped referring to the size, because it can be misleading about the actual performance of the model."

The Transform Technology Summits start October 13th with Low-Code/No Code: Enabling Enterprise Agility. In January 2020, OpenAI laid out the scaling law of language models: You can improve the performance of any neural language model by adding more training data, more model parameters, and more compute. Since then, there has been an arms race to train ever larger neural networks for natural language processing (NLP). And the latest to join the list is AI21 with its 178 billion parameter model. Before this, Amnon founded Mobileye, the NYSE-listed self-driving tech company that Intel acquired for $15.4 billion.

Such is just one of the attributes of Jurassic, a computer program introduced Wednesday by Tel Aviv-based artificial intelligence startup AI21 Labs. GPT-3, of course, is the language program from the San Francisco-based startup OpenAI that rocked the world in 2020 by generating sentences and whole articles that seemed quite human-like. GPT-3 also shocked the world by being kept inside a fairly restrictive beta testing arrangement by OpenAI. AI21 is promising to go OpenAI not one better, but two better, with what it claims are superior benchmark results on a test known as « few shot learning, » and a more open program for beta testers. On the latter score, AI21 is making development use of the program available as an « open beta, » it said, where anyone can sign up to use the program and there is « no wait list.