The Domain Mixed Unit: A New Neural Arithmetic Layer

The Domain Mixed Unit (DMU) is a new neural arithmetic unit that learns a single parameter gate G that mixes a state between log-space and linear-space representations while performing either addition (DMU add) or subtraction (DMU sub) in said space. These are the two initializations proposed for the DMU: one covering addition and multiplication, and another covering subtraction and division. The DMU achieves state-of-the-art performance on the NALM Benchmark, a dataset designed to test the ability of neural arithmetic units to generalize arithmetic operations, specifically performing with the highest percentage solved over all seeds on multiplication and division. Neural Arithmetic Units (NAUs) are specialized sub-units or networks designed to interpretably represent arithmetic operations while maintaining differentiability, allowing gradients to flow through them during training. These units can be integrated into larger neural architectures to provide explicit arithmetic capabilities.