Language models (LMs) exhibit remarkable abilities to solve new tasks from just a few examples or textual instructions, especially at scale. They also, paradoxically, struggle with basic functionality, such as arithmetic or factual lookup, where much simpler and smaller specialized models excel. In this paper, we show that LMs can teach themselves to use external tools via simple APIs and achieve the best of both worlds. We introduce Toolformer, a model trained to decide which APIs to call, when to call them, what arguments to pass, and how to best incorporate the results into future token prediction. This is done in a self-supervised way, requiring nothing more than a handful of demonstrations for each API. We incorporate a range of tools, including a calculator, a Q&A system, a search engine, a translation system, and a calendar. Toolformer achieves substantially improved zero-shot performance across a variety of downstream tasks, often competitive with much larger models, without sacrificing its core language modeling abilities.

Fecal vandal's nearly weeklong crime spree comes to an end when police catch her in the act Catching the horny landlady teaching your boyfriend mouth-to-mouth is not a sign that it's time to move MAGA bikini congresswoman sends a message to big brother, Dale Earnhardt turns 75 & MLB fan gets pulverized! Wait... Who is actually using highway rest stop BBQ grills? Hilary Duff's latest Instagram content has suburban millennial moms gasping, a tennis match turns nasty & MEAT Opening day at Six Flags St. Louis ended in chaos after brawl with as many as 100 people broke out Mountain climber survives terrifying 500-foot fall in California's Sierra Nevada, night stranded on ledge Ella Langley's brand deal with American Eagle shows Bud Light how it could've been in 2023, fan fight & MEAT Shannon Elizabeth, to nobody's surprise, cashes in on OnlyFans with reported 7-figure payday in her first week Airline doesn't buy couple's claim that they were praying, bans them for attempting to join mile high club Bill Maher & David Cross get into heated war of words over'looney left' & trans rights, including 3-year-old'Map wars': Brit Hume says redistricting battle is'as bitter' as he's ever seen it Candidates make their case as California governor's race intensifies Hegseth, Caine defend Pentagon's budget request on Capitol Hill Greg Gutfeld: Walz tries to appear'above it all,' but is'drowning' in corruption Ukraine is'militarily' defeated: Trump Trump posts AI image of himself with a gun, says Iran'better get smart soon' Trump calls Comey a'dirty cop' and a'crooked man' Sen. Rand Paul backs White House ballroom after WHCA shooting Steven Hilton says voter ID push could boost GOP turnout in California governor's race There's no question that robots are going to be coming for some folks' jobs sooner rather than later, and it looks like baggage handlers could be one of the first on the robo-chopping block. Japan Airlines is going to start rolling out its humanoid robots to help with baggage at Tokyo's Haneda Airport. Now, while I'm usually not one to celebrate something like this -- I feel it's just one step closer to all of us having to pay our respects to robot overlords -- I was excited about it.

Protein inverse folding has attracted increasing attention in recent years. However, we observe that current methods are usually limited to the CATH dataset and the recovery metric. The lack of a unified framework for ensembling and comparing different methods hinders the comprehensive investigation. In this paper, we propose ProteinInvBench, a new benchmark for protein design, which comprises extended protein design tasks, integrated models, and diverse evaluation metrics. We broaden the application of methods originally designed for single-chain protein design to new scenarios of multi-chain and de novo protein design. Recent impressive methods, including GraphTrans, StructGNN, GVP, GCA, AlphaDesign, ProteinMPNN, PiFold and KWDesign are integrated into our framework. In addition to the recovery, we also evaluate the confidence, diversity, sc-TM, efficiency, and robustness to thoroughly revisit current protein design approaches and inspire future work. As a result, we establish the first comprehensive benchmark for protein design, which is publicly available at https://github.com/A4Bio/OpenCPD.

We introduce general tools for designing efficient private estimation algorithms, in the high-dimensional settings, whose statistical guarantees almost match those of the best known non-private algorithms. To illustrate our techniques, we consider two problems: recovery of stochastic block models and learning mixtures of spherical Gaussians. For the former, we present the first efficient (ε,δ)-differentially private algorithms for both weak recovery and exact recovery. Previously known algorithms achieving comparable guarantees required quasi-polynomial time. We complement these results with an information-theoretic lower bound that highlights how the guarantees of our algorithms are almost tight. For the latter, we design an (ε,δ)-differentially private algorithm that recovers the centers of the k-mixture when the minimum separation is at least O(k1/t t). For all choices of t, this algorithm requires sample complexity n kO(1)dO(t) and time complexity (nd)O(t). Prior work required either an additional additive Ω( logn) term in the minimum separation or an explicit upper bound on the Euclidean norm of the centers.

Dataset condensation (DC) distills a large real-world dataset into a small synthetic dataset, with the goal of training a network from scratch on the latter that performs similarly to the former. State-of-the-art (SOTA) DC methods have achieved satisfactory results through techniques such as accuracy, gradient, training trajectory, or distribution matching. However, these works all perform matching in the high-dimension pixel space, ignoring that natural images are usually locally connected and have lower intrinsic dimensions, resulting in low condensation efficiency. In this work, we propose a simple-yet-efficient dataset condensation plugin that matches the raw and synthetic datasets in a low-dimensional manifold.