compact
In 2024, the camera of the year was a drone
Aside from the global shutter on Sony's A9 III and some cool mirrorless options -- the Fujifilm X100 VI, Panasonic S9 and Canon EOS R5 II come to mind -- 2024 was a dull year for cameras full of small tweaks and minor improvements. For 200, aerial photography is now finally in reach for just about anyone. DJI released its product lineup this year with a sword of Damocles hanging over its head: the US government was planning to ban sales of the company's products by the end of 2024 over potential fears of spying. It was only at the last minute that DJI gained a reprieve, thanks in large part to lobbying by public safety groups that heavily rely on its drones. It now has until the end of 2025 to prove that its products don't pose a risk.
CompAct: Compressed Activations for Memory-Efficient LLM Training
Shamshoum, Yara, Hodos, Nitzan, Sieradzki, Yuval, Schuster, Assaf
We introduce CompAct, a technique that reduces peak memory utilization on GPU by 25-30% for pretraining and 50% for fine-tuning of LLMs. Peak device memory is a major limiting factor in training LLMs, with various recent works aiming to reduce model memory. However most works don't target the largest component of allocated memory during training: the model's compute graph, which is stored for the backward pass. By storing low-rank, Figure 1: Breakdown of memory components for various compressed activations to be used in the backward LLaMA model sizes, with batch size 256. Blue: pass we greatly reduce the required memory, linear operations compressed by CompAct; Red: nonlinear unlike previous methods which only reduce operations which CompAct doesn't compress; optimizer overheads or the number of Green: model parameters and non-linear operation's optimizer trained parameters. Our compression uses random states. Most of the memory is used by the computational projection matrices, thus avoiding additional graph. CompAct's compression gets more memory overheads.
CompAct: Compressing Retrieved Documents Actively for Question Answering
Yoon, Chanwoong, Lee, Taewhoo, Hwang, Hyeon, Jeong, Minbyul, Kang, Jaewoo
Retrieval-augmented generation supports language models to strengthen their factual groundings by providing external contexts. However, language models often face challenges when given extensive information, diminishing their effectiveness in solving questions. Context compression tackles this issue by filtering out irrelevant information, but current methods still struggle in realistic scenarios where crucial information cannot be captured with a single-step approach. To overcome this limitation, we introduce CompAct, a novel framework that employs an active strategy to condense extensive documents without losing key information. Our experiments demonstrate that CompAct brings significant improvements in both performance and compression rate on multi-hop question-answering (QA) benchmarks. CompAct flexibly operates as a cost-efficient plug-in module with various off-the-shelf retrievers or readers, achieving exceptionally high compression rates (47x).
Compact: Approximating Complex Activation Functions for Secure Computation
Islam, Mazharul, Arora, Sunpreet S., Chatterjee, Rahul, Rindal, Peter, Shirvanian, Maliheh
Secure multi-party computation (MPC) techniques can be used to provide data privacy when users query deep neural network (DNN) models hosted on a public cloud. State-of-the-art MPC techniques can be directly leveraged for DNN models that use simple activation functions (AFs) such as ReLU. However, DNN model architectures designed for cutting-edge applications often use complex and highly non-linear AFs. Designing efficient MPC techniques for such complex AFs is an open problem. Towards this, we propose Compact, which produces piece-wise polynomial approximations of complex AFs to enable their efficient use with state-of-the-art MPC techniques. Compact neither requires nor imposes any restriction on model training and results in near-identical model accuracy. We extensively evaluate Compact on four different machine-learning tasks with DNN architectures that use popular complex AFs SiLU, GeLU, and Mish. Our experimental results show that Compact incurs negligible accuracy loss compared to DNN-specific approaches for handling complex non-linear AFs. We also incorporate Compact in two state-of-the-art MPC libraries for privacy-preserving inference and demonstrate that Compact provides 2x-5x speedup in computation compared to the state-of-the-art approximation approach for non-linear functions -- while providing similar or better accuracy for DNN models with large number of hidden layers
The Future of Data Analytics - Compact
A global KPMG survey showed that organizations cannot yet reap all the benefits of data analytics due to data quality issues and a lack of capable resources. In 30 years' time, developments in data analytics itself could solve this issue, making many current professions in the sector obsolete. The impossible will become possible, and this may well lead to an autonomous decision-making process. Data analytics is expected to radically change the way we live and do business in the future. Already today we use the analytics in our technology devices, for many decisions in our lives.
Test Drive: Can the 2021 Polestar 2 electric sedan outshine Tesla?
The Polestar 2 is the first all-electric car from Polestar, a new brand owned by Volvo. Fox News Autos Gary Gastelu takes it for a spin. Well, let's go ahead and add one more, anyway. It's called Polestar and it's a little different than the rest because it's a standalone subsidiary of Volvo, so comes to the table with some experience. The name originally belonged to a Swedish outfit called Polestar Racing that built competition Volvos and modified the company's sedate street cars into high performance machines.
7 Best Desktop PCs for Gaming (2020): Compact, Custom, Cheap
I play most of my games the same way I read a good paperback: Curled up on the couch or in bed. I mean, where else can we go right now? I'm usually glued to my Nintendo Switch or laptop; they're great for everyday gaming. But sometimes I get a craving for something a little more immersive, a gaming experience with razor-sharp graphics and a buttery-smooth refresh rate. That's where a desktop comes in handy.
Enriching Documents with Compact, Representative, Relevant Knowledge Graphs
Li, Shuxin, Huang, Zixian, Cheng, Gong, Kharlamov, Evgeny, Gunaratna, Kalpa
A prominent application of knowledge graph (KG) is document enrichment. Existing methods identify mentions of entities in a background KG and enrich documents with entity types and direct relations. We compute an entity relation subgraph (ERG) that can more expressively represent indirect relations among a set of mentioned entities. To find compact, representative, and relevant ERGs for effective enrichment, we propose an efficient best-first search algorithm to solve a new combinatorial optimization problem that achieves a trade-off between representativeness and compactness, and then we exploit ontological knowledge to rank ERGs by entity-based document-KG and intra-KG relevance. Extensive experiments and user studies show the promising performance of our approach.
JBL Flip 4 Bluetooth speaker review: Better-than-expected sound in a compact, waterproof enclosure
There's nothing quite like feeling as though you got more than what you paid for. That's how you'll feel if you drop a C-note on JBL's Flip 4 Bluetooth speaker. Small, easy tote, and waterproof to the point of being submersible, this speaker sounds like it's considerably bigger and much more expensive. It can easily fill a large bedroom with sound, and it can also hold its own outside. But the Flip 4 is more than just loud.