hvt
Hierarchical Verification of Speculative Beams for Accelerating LLM Inference
Sen, Jaydip, Puvvala, Harshitha, Dasgupta, Subhasis
Large language models (LLMs) have achieved remarkable success across diverse natural language processing tasks but face persistent challenges in inference efficiency due to their autoregressive nature. While speculative decoding and beam sampling offer notable improvements, traditional methods verify draft sequences sequentially without prioritization, leading to unnecessary computational overhead. This work proposes the Hierarchical Verification Tree (HVT), a novel framework that restructures speculative beam decoding by prioritizing high-likelihood drafts and enabling early pruning of suboptimal candidates. Theoretical foundations and a formal verification-pruning algorithm are developed to ensure correctness and efficiency. Integration with standard LLM inference pipelines is achieved without requiring retraining or architecture modification. Experimental evaluations across multiple datasets and models demonstrate that HVT consistently outperforms existing speculative decoding schemes, achieving substantial reductions in inference time and energy consumption while maintaining or enhancing output quality. The findings highlight the potential of hierarchical verification strategies as a new direction for accelerating large language model inference.
- North America > United States > Missouri (0.04)
- Europe > Switzerland (0.04)
- Asia > India > West Bengal > Kolkata (0.04)
Abductive Inference for Combat: Using SCARE-S2 to Find High-Value Targets in Afghanistan
Shakarian, Paulo (U.S. Army) | Nagel, Mago (University of Maryland) | Schuetzle, Brittany (University of Maryland) | Subrahmanian, V.S. (University of Maryland)
Recently, geospatial abduction was introduced by the authors in [Shakarian et. al. 2010] as a way to infer unobserved geographic phenomena from a set of known observations and constraints between the two. In this paper, we introduce the SCARE-S2 software tool which applies geospatial abduction to the environment of Afghanistan. Unlike previous work, where we looked for small weapon caches supporting local attacks, here we look for insurgent high-value targets (HVT's), supporting insurgent operations in two provinces. These HVT's include the locations of insurgent leaders and major supply depots. Applying this method of inference to Afghanistan introduces several practical issues not addressed in previous work. Namely, we are conducting inference in a much larger area (24,940 sq km as compared to 675 sq km in previous work), on more varied terrain, and must consider the influence of many local tribes. We address all of these problems and evaluate our software on 6 months of real-world counter-insurgency data. We show that we are able to abduce regions of a relatively small area (on average, under 100 sq km and each containing, on average, 4.8 villages) that are more dense with HVT's (35 X more than the overall area considered).
- North America > United States > Maryland > Prince George's County > College Park (0.14)
- Asia > Middle East > Iraq > Baghdad Governorate > Baghdad (0.05)
- Asia > Afghanistan > Kandahar Province > Kandahar (0.05)
- (6 more...)
- Law Enforcement & Public Safety (1.00)
- Government > Regional Government > North America Government > United States Government (0.47)
- Government > Military > Army (0.47)