We introduce Gemma 3, a multimodal addition to the Gemma family of lightweight open models, ranging in scale from 1 to 27 billion parameters. This version introduces vision understanding abilities, a wider coverage of languages and longer context - at least 128K tokens. We also change the architecture of the model to reduce the KV-cache memory that tends to explode with long context. This is achieved by increasing the ratio of local to global attention layers, and keeping the span on local attention short. The Gemma 3 models are trained with distillation and achieve superior performance to Gemma 2 for both pre-trained and instruction finetuned versions. In particular, our novel post-training recipe significantly improves the math, chat, instruction-following and multilingual abilities, making Gemma3-4B-IT competitive with Gemma2-27B-IT and Gemma3-27B-IT comparable to Gemini-1.5-Pro across benchmarks. We release all our models to the community.

Large language models (LLMs) are increasingly deployed in everyday applications, demanding robust general reasoning capabilities and diverse reasoning skillset. However, current LLM reasoning benchmarks predominantly focus on mathematical and coding abilities, leaving a gap in evaluating broader reasoning proficiencies. One particular exception is the BIG-Bench dataset, which has served as a crucial benchmark for evaluating the general reasoning capabilities of LLMs, thanks to its diverse set of challenging tasks that allowed for a comprehensive assessment of general reasoning across various skills within a unified framework. However, recent advances in LLMs have led to saturation on BIG-Bench, and its harder version BIG-Bench Hard (BBH). State-of-the-art models achieve near-perfect scores on many tasks in BBH, thus diminishing its utility. To address this limitation, we introduce BIG-Bench Extra Hard (BBEH), a new benchmark designed to push the boundaries of LLM reasoning evaluation. BBEH replaces each task in BBH with a novel task that probes a similar reasoning capability but exhibits significantly increased difficulty. We evaluate various models on BBEH and observe a (harmonic) average accuracy of 9.8\% for the best general-purpose model and 44.8\% for the best reasoning-specialized model, indicating substantial room for improvement and highlighting the ongoing challenge of achieving robust general reasoning in LLMs. We release BBEH publicly at: https://github.com/google-deepmind/bbeh.

Logistic regression is a commonly used representation for aggregators in Bayesian belief networks when a child has multiple parents. In this paper we consider extending logistic regression to relational models, where we want to model varying populations and interactions among parents. In this paper, we first examine the representational problems caused by population variation. We show how these problems arise even in simple cases with a single parametrized parent, and propose a linear relational logistic regression which we show can represent arbitrary linear (in population size) decision thresholds, whereas the traditional logistic regression cannot. Then we examine representing interactions among the parents of a child node, and representing non-linear dependency on population size.

A few years ago this month, Portland, Oregon artist Darius Kazemi watched a flood of tweets from would-be novelists. November is National Novel Writing Month, a time when people hunker down to churn out 50,000 words in a span of weeks. To Kazemi, a computational artist whose preferred medium is the Twitter bot, the idea sounded mildly tortuous. "I was thinking I would never do that," he says. "But if a computer could do it for me, I'd give it a shot."

In an effort to make its apps more conversational, Google fed its AI engine a whopping 2,865 romance novels so it can improve its understanding of language. The idea is to improve the way Google products respond to users. Software engineer Andrew Dai, who led the project, told BuzzFeed News that this sort of work could help make the responses from the company's search app, as well as the'Smart Reply' feature in Inbox, more natural and varied. Our biggest ever edition of TNW Conference is fast approaching! Dai added that romance novels are great for training AI because they mostly follow the same plot – allowing the AI to focus on picking up nuances of language.

At Facebook's F8 conference in Silicon Valley, David Marcus, the company's head of messaging, proudly demonstrated its new suite of chatbots. Users can now get in a conversation with the likes of CNN, H&M, and HP, and ask for help shopping, or the latest headlines. The chatbots aren't very good, but that doesn't mean Facebook isn't proud of them anyway: "I guarantee you're going to spend way more money than you want on this," Marcus chuckled on stage. But even though Facebook might want to sell itself as the pioneer of chatbots, the real leaders in the field aren't working in the AI research teams of silicon valley; they're collaborating at events like last week's BotSummit in the V&A, or this weekend's Art of Bots exhibition in Somerset House. Move over chatbots: it's time to meet the artbots. BotSummit, now in its fourth year but held outside the US for the first time, is the creation of internet artist Darius Kazemi, whose medium he describes as "bots and generators and other weird internet stuff".

A day after Microsoft launched its "AI teen girl Twitter chatbot," Twitter taught her to be racist. The thing is, this was all very much preventable. I talked to some creators of Twitter bots about @TayandYou, and the consensus was that Microsoft had fallen far below the baseline of ethical botmaking. "The makers of @TayandYou absolutely 10000 percent should have known better," thricedotted, a veteran Twitter botmaker and natural language processing researcher, told me via email. "It seems like the makers of @TayandYou attempted to account for a few specific mishaps, but sorely underestimated the vast potential for people to be assholes on the internet."