Goto

Collaborating Authors

Accumulation Bit-Width Scaling For Ultra-Low Precision Training Of Deep Networks

arXiv.org Machine Learning

Efforts to reduce the numerical precision of computations in deep learning training have yielded systems that aggressively quantize weights and activations, yet employ wide high-precision accumulators for partial sums in inner-product operations to preserve the quality of convergence. The absence of any framework to analyze the precision requirements of partial sum accumulations results in conservative design choices. This imposes an upper-bound on the reduction of complexity of multiply-accumulate units. We present a statistical approach to analyze the impact of reduced accumulation precision on deep learning training. Observing that a bad choice for accumulation precision results in loss of information that manifests itself as a reduction in variance in an ensemble of partial sums, we derive a set of equations that relate this variance to the length of accumulation and the minimum number of bits needed for accumulation. We apply our analysis to three benchmark networks: CIFAR-10 ResNet 32, ImageNet ResNet 18 and ImageNet AlexNet. In each case, with accumulation precision set in accordance with our proposed equations, the networks successfully converge to the single precision floating-point baseline. We also show that reducing accumulation precision further degrades the quality of the trained network, proving that our equations produce tight bounds. Overall this analysis enables precise tailoring of computation hardware to the application, yielding area- and power-optimal systems.


Apples surviving in a thirsty landscape

Science

Apple production in the semiarid Loess Plateau in China is susceptible to water deficiencies. Sun et al. have engineered an avenue for drought resistance into apple trees. Overexpression of the apple (Malus domestica) gene MdATG18a (encoding autophagy-related protein 18a), which is normally up-regulated in response to drought, allowed transgenic plants to better survive a dry spell. The amount of cellular damage, as measured by, for example, membrane integrity and accumulation of reactive oxygen species, was reduced. The authors suggest that the improved intracellular physiology left by hypervigilant autophagy systems allowed the cells to survive suboptimal conditions.


Calibrating mtDNA mutations through age

Science

Mutation Mitochondrial DNA (mtDNA) is a separate genome found in eukaryotic cells that is maternally inherited. Mutations in mtDNA underlie several human diseases, and the accumulation of these mutations has been associated with aging. Arbeithuber et al. used duplex sequencing to trace accumulation of spontaneous mtDNA mutations in oocytes, brain, and muscle cells of mice. Ten-month-old mothers showed a two- to threefold increased rate of mtDNA mutation compared with their 1-month-old pups. The authors found that the D-loop, a stretch of triple-stranded highly variable DNA in the noncoding region of the circular mtDNA where replication initiates, accumulated the most mutations. These mtDNA mutations occurred in patterns, indicating that they were caused by replication errors. It is possible that inheritance of aged mtDNA from older mothers may have health consequences for their offspring. PLOS BIOL. 18 , e3000745 (2020).


Matchbox mania

BBC News

I have been collecting matchboxes seriously since 2012. My sources include collectors all over the world, dealers, auctions, flea markets, and just about any place I can think of. The accumulation, as I like to call it, includes all kinds of material related to the Indian matchbox industry. Within this rapidly growing accumulation, I often come across labels and subjects that I get curious about and that is how my collection takes a thematic approach. My recent exhibition, titled "Matchbox Labels And The Stories They Tell", features some prominent trends and themes spanning the entire history of the matchbox industry.


More Snow on the Way for Maine, New Hampshire

U.S. News

Meteorologist Chris Legro said the heaviest accumulations of up to 6 inches will be along the coast in southern New Hampshire and southernmost Maine, and in Downeast Maine. He said accumulations of 2 to 4 inches are anticipated in between those two regions.