The deployment of large language models (LLMs) is frequently hindered by prohibitive memory and computational requirements. While quantization mitigates these bottlenecks, maintaining model fidelity in the sub-1-bit regime remains a persistent challenge. In this paper, we introduce LITTLEBIT, a novel framework for extreme LLM compression. We target quantization rates as low as 0.1bits per weight (BPW), achieving a memory reduction of approximately 31, which effectively compresses Llama2-13B to under 0.9GB. We represent weights via low-rank latent matrix factorization and subsequently binarize the resulting factors. To counteract the information loss inherent to such drastic precision reduction, we integrate a multi-scale compensation mechanism that learns importance parameters across row, column, and latent dimensions. Two primary contributions enable effective training: Dual Sign-Value-Independent Decomposition (Dual-SVID) for quantization-aware training (QAT) initialization, and Residual Compensation to minimize approximation errors. Extensive experiments confirm the superiority of LITTLEBIT in the sub-1-bit domain; for instance, our method at 0.1 BPW surpasses the performance of leading techniques operating at 0.7BPW on Llama2-7B. We establish a new sizeperformance trade-off--unlocking a potential 11.6 inference speedup relative to FP16--and render powerful LLMs practical for resource-constrained environments.

Welcome to TechCrunch's 2019 Holiday Gift Guide! Need help with gift ideas? We'll be rolling out gift guides from now through the end of December, so check back regularly. We've refreshed our annual STEM toy gift guide with the latest wares clamoring to entice and inspire kids with coding tricks and electronic wizardry. But lean in to this market and you'll find a number of STEM toy makers have winked out of existence since this time last year, or else been folded into others' empires. Such as littleBits selling to Sphero this fall, or Root Robotics being picked up by robot vac giant iRobot in June.

Every Christmas in the '80s, I wanted the same thing as many other pint-sized Star Wars fans: a robot sidekick to call my own. And not just any old droid would do: It had to be an R2-D2, specifically one that could drop its third leg down and cruise around the world at my side. Growing up in the Death Star era, our entire generation thought it had "The Force." But eventually we realized that moving objects with our thoughts and duping people with Jedi mind tricks were all in our imaginations. But droids--they were real, or at least they could be, one day.