Quantization has emerged as a promising technique for improving the memory and computational efficiency of large language models (LLMs). Though the trade-off between performance and efficiency is well-known, there is still much to be learned about the relationship between quantization and LLM performance. To shed light on this relationship, we propose a new perspective on quantization, viewing it as perturbations added to the weights and activations of LLMs. We call this approach "the lens of perturbation". Using this lens, we conduct experiments with various artificial perturbations to explore their impact on LLM performance. Our findings reveal several connections between the properties of perturbations and LLM performance, providing insights into the failure cases of uniform quantization and suggesting potential solutions to improve the robustness of LLM quantization. To demonstrate the significance of our findings, we implement a simple non-uniform quantization approach based on our insights. Our experiments show that this approach achieves minimal performance degradation on both 4-bit weight quantization and 8-bit quantization for weights and activations. These results validate the correctness of our approach and highlight its potential to improve the efficiency of LLMs without sacrificing performance.

In a metal workshop in the industrial city of Kryvyi Rih in southern Ukraine, a homemade anti-drone system waits to be mounted on a military pick-up truck. The contraption -- a heavy machine gun welded to steel tubes -- is one of several do-it-yourself weapons that are proving to be valuable additions to the Ukraine war effort. "We have the skills and the equipment, and we don't lack ideas," said Sergey Bondarenko in the workshop near the southern front. The well-built 39-year-old with a long black beard is a local leader of the territorial defence, a unit of the Ukrainian army. The device will be accompanied by shock absorbers, for more stability and precision, Bondarenko told AFP beside the anti-drone prototype.

It may not appear so but Vladimir Bondarenko, a Ukrainian blogger, has a lot in common with Keenan Ramsey, a "growth specialist" at Silicon Valley communications firm RingCentral. Bondarenko is a former aviation engineer with a square jaw, close-cut brown hair and grey eyes who apparently took to writing anti-government screeds after Ukraine's aviation industry "collapsed". Ramsey, with her straight, pearly teeth, brown hair that falls past her shoulders and a flawless complexion, studied at New York University and is a fan of Bill Gates's ex-wife Melinda. Keenan and Vladimir are both fakes: digital faces generated by artificial intelligence software. Facebook discovered -- and eliminated -- the Bondarenko account in February, tracing it to a Russian troll farm.