binary variable
Binary Expansion Group Intersection Network
Conditional independence is central to modern statistics, but beyond special parametric families it rarely admits an exact covariance characterization. We introduce the binary expansion group intersection network (BEGIN), a distribution-free graphical representation for multivariate binary data and bit-encoded multinomial variables. For arbitrary binary random vectors and bit representations of multinomial variables, we prove that conditional independence is equivalent to a sparse linear representation of conditional expectations, to a block factorization of the corresponding interaction covariance matrix, and to block diagonality of an associated generalized Schur complement. The resulting graph is indexed by the intersection of multiplicative groups of binary interactions, yielding an analogue of Gaussian graphical modeling beyond the Gaussian setting. This viewpoint treats data bits as atoms and local BEGIN molecules as building blocks for large Markov random fields. We also show how dyadic bit representations allow BEGIN to approximate conditional independence for general random vectors under mild regularity conditions. A key technical device is the Hadamard prism, a linear map that links interaction covariances to group structure.
Optimization of the quantization of dense neural networks from an exact QUBO formulation
Subiรฑas, Sergio Muรฑiz, Gonzรกlez, Manuel L., Gรณmez, Jorge Ruiz, Ali, Alejandro Mata, Martรญn, Jorge Martรญnez, Hernando, Miguel Franco, Garcรญa-Vico, รngel Miguel
This work introduces a post-training quantization (PTQ) method for dense neural networks via a novel ADAROUND-based QUBO formulation. Using the Frobenius distance between the theoretical output and the dequantized output (before the activation function) as the objective, an explicit QUBO whose binary variables represent the rounding choice for each weight and bias is obtained. Additionally, by exploiting the structure of the coefficient QUBO matrix, the global problem can be exactly decomposed into $n$ independent subproblems of size $f+1$, which can be efficiently solved using some heuristics such as simulated annealing. The approach is evaluated on MNIST, Fashion-MNIST, EMNIST, and CIFAR-10 across integer precisions from int8 to int1 and compared with a round-to-nearest traditional quantization methodology.
OptPipe: Memory- and Scheduling-Optimized Pipeline Parallelism for LLM Training
Li, Hongpei, Zhang, Han, Liu, Huikang, Ge, Dongdong, Ye, Yinyu
Pipeline parallelism (PP) has become a standard technique for scaling large language model (LLM) training across multiple devices. However, despite recent progress in reducing memory consumption through activation offloading, existing approaches remain largely heuristic and coarse-grained, often overlooking the fine-grained trade-offs between memory, computation, and scheduling latency. In this work, we revisit the pipeline scheduling problem from a principled optimization perspective. We observe that prevailing strategies either rely on static rules or aggressively offload activations without fully leveraging the interaction between memory constraints and scheduling efficiency. To address this, we formulate scheduling as a constrained optimization problem that jointly accounts for memory capacity, activation reuse, and pipeline bubble minimization. Solving this model yields fine-grained schedules that reduce pipeline bubbles while adhering to strict memory budgets. Our approach complements existing offloading techniques: whereas prior approaches trade memory for time in a fixed pattern, we dynamically optimize the tradeoff with respect to model structure and hardware configuration. Experimental results demonstrate that our method consistently improves both throughput and memory utilization. In particular, we reduce idle pipeline time by up to 50% under the same per-device memory limit, and in some cases, enable the training of larger models within limited memory budgets.
A MILP-Based Solution to Multi-Agent Motion Planning and Collision Avoidance in Constrained Environments
Jaitly, Akshay, Cline, Jack, Farzan, Siavash
We propose a mixed-integer linear program (MILP) for multi-agent motion planning that embeds Polytopic Action-based Motion Planning (PAAMP) into a sequence-then-solve pipeline. Region sequences confine each agent to adjacent convex polytopes, while a big-M hyperplane model enforces inter-agent separation. Collision constraints are applied only to agents sharing or neighboring a region, which reduces binary variables exponentially compared with naive formulations. An L1 path-length-plus-acceleration cost yields smooth trajectories. We prove finite-time convergence and demonstrate on representative multi-agent scenarios with obstacles that our formulation produces collision-free trajectories an order of magnitude faster than an unstructured MILP baseline.
Accelerating Signal-Temporal-Logic-Based Task and Motion Planning of Bipedal Navigation using Benders Decomposition
Ren, Jiming, Lin, Xuan, Mineyev, Roman, Feigh, Karen M., Coogan, Samuel, Zhao, Ye
--T ask and motion planning under Signal T emporal Logic constraints is known to be NP-hard. A common class of approaches formulates these hybrid problems, which involve discrete task scheduling and continuous motion planning, as mixed-integer programs (MIP). However, in applications for bipedal locomotion, introduction of non-convex constraints such as kinematic reachability and footstep rotation exacerbates the computational complexity of MIPs. In this work, we present a method based on Benders Decomposition to address scenarios where solving the entire monolithic optimization problem is prohibitively intractable. Benders Decomposition proposes an iterative cutting-plane technique that partitions the problem into a master problem to prototype a plan that meets the task specification, and a series of subproblems for kinematics and dynamics feasibility checks. Our experiments demonstrate that this method achieves faster planning compared to alternative algorithms for solving the resulting optimization program with nonlinear constraints. A project website can be found at http: //bipedal-stl.github.io/. Note to Practitioners -- Bipedal robots are increasingly demanded in warehouses and factories for complex automation tasks such as stacking, delivering, and interacting with other robots under strict time and safety constraints. However, planning such operations under formal language instructions such as Signal T emporal Logic (STL) specifications often results in large-scale mixed-integer programs that are impractical to be solved in a timely manner . This paper introduces an accelerated task and motion planning (T AMP) approach via Benders Decomposition that splits the task into a high-level scheduling problem and lower-level motion feasibility checks, allowing practitioners to find feasible and optimal task and motion plans far more efficiently. Compared to conventional monolithic solvers or alternative decomposition methods, our approach can generate solutions more than twenty times faster while rigorously satisfying kinematic and dynamic constraints. Benchmark scenarios, including factory delivery and warehouse logistics, demonstrate how our method handles realistic automation scenarios involving long planning horizons and complicated task specifications.