Goto

Collaborating Authors

 hess




Model-Preserving Adaptive Rounding

Tseng, Albert, Sun, Zhaofeng, De Sa, Christopher

arXiv.org Artificial Intelligence

The goal of quantization is to produce a compressed model whose output distribution is as close to the original model's as possible. To do this tractably, most quantization algorithms minimize the immediate activation error of each layer as a proxy for the end-to-end error. However, this ignores the effect of future layers, making it a poor proxy. In this work, we introduce Yet Another Quantization Algorithm (YAQA), an adaptive rounding algorithm that directly considers the error at the network's output. YAQA introduces a series of theoretical results that culminate in the first end-to-end error bounds for quantization algorithms. First, we characterize the convergence time of adaptive rounding algorithms via the structure of their Hessian approximations. We then show that the end-to-end error can be bounded by the approximation's cosine similarity to the true Hessian. This admits a natural Kronecker-factored approximation with corresponding near-optimal Hessian sketches. YAQA is provably better than GPTQ/LDLQ and empirically reduces the error by $\approx 30\%$ over these methods. YAQA even achieves a lower error than quantization aware training. This translates to state of the art performance on downstream tasks, all while adding no inference overhead.


"A Minecraft Movie" Is a Tale of Two Cinematic Universes

The New Yorker

I've never played Minecraft in my life--but then I'm not a Christian, either, and have always delighted in the distinctly Mormon cinematic universe of Jared Hess, the director of "A Minecraft Movie." He's best known for "Napoleon Dynamite," from 2004, which evokes its spiritual milieu only implicitly, by the absence of secular pop culture and of teen-age ribaldry. He followed it with "Nacho Libre," starring Jack Black as a friar who enters the wrestling ring to save a convent, and, in 2009, with "Gentlemen Broncos," a celestial gross-out vision of an adolescent gospel. His satire "Don Verdean," from 2015, is explicitly set in church communities and involves relic smuggling in Israel; his 2016 comedy, "Masterminds," is a heist film that's centered on grace and holy innocence. With "A Minecraft Movie," I was impatient to see what Hess would do with another world of extreme fantasy, akin to that of "Gentlemen Broncos." The short answer is, too much and not nearly enough; the I.P. is the boss, the characters are its minions, and Hess--constrained both by a script that he didn't write and by the demands of complex C.G.I.--struggles to live up to his own œuvre, which is among the most substantially loopy (or loopily substantial) in modern cinema.

  Country: Asia > Middle East > Israel (0.25)
  Industry:

Block-busted: why homemade Minecraft movies are the real hits

The Guardian

By any estimation, Minecraft is impossibly successful. The bestselling video game ever, as of last December it had 204 million monthly active players. Since it was first released in 2011, it has generated over 3bn ( 2.3bn) in revenue. What's more, its players have always been eager to demonstrate their fandom outside the boundaries of the game itself. In 2021, YouTube calculated that videos related to the game – tutorials, walk-throughs, homages, parodies – had collectively been viewed 1tn times. In short, it is a phenomenon.


A Minecraft Movie review: It's good, actually

Engadget

I too rolled my eyes when A Minecraft Movie was announced. We're all tired of seeing Jack Black in video game movies -- he was fine in Super Mario Bros., but good god Borderlands was a disaster. And the Minecraft film's trailers did it no favors, another soulless movie produced on a virtual set about a game that's completely open-ended and plotless. But it turns out A Minecraft Movie is actually good. Honestly, I'm as surprised as you are.


Adaptive Stochastic Gradient Descents on Manifolds with an Application on Weighted Low-Rank Approximation

Yang, Peiqi, Xu, Conglong, Wu, Hao

arXiv.org Artificial Intelligence

We prove a convergence theorem for stochastic gradient descents on manifolds with adaptive learning rate and apply it to the weighted low-rank approximation problem.


How to have a child in the digital age

MIT Technology Review

But how do we retain control over our bodies when corporations and the medical establishment have access to our most personal information? What happens when humans stop relying on their village, or even their family, for advice on having a kid and instead go online, where there's a constant onslaught of information? How do we make sense of the contradictions of the internet--the tension between what's inherently artificial and the "natural" methods its denizens are so eager to promote? In her new book, Second Life: Having a Child in the Digital Age (Doubleday, 2025), Hess explores these questions while delving into her firsthand experiences with apps, products, algorithms, online forums, advertisers, and more--each promising an easier, healthier, better path to parenthood. After welcoming her son, who is now healthy, in 2020 and another in 2022, Hess is the perfect person to ask: Is that really what they're delivering?


Bayesian Online Natural Gradient (BONG)

Jones, Matt, Chang, Peter, Murphy, Kevin

arXiv.org Machine Learning

We propose a novel approach to sequential Bayesian inference based on variational Bayes. The key insight is that, in the online setting, we do not need to add the KL term to regularize to the prior (which comes from the posterior at the previous timestep); instead we can optimize just the expected log-likelihood, performing a single step of natural gradient descent starting at the prior predictive. We prove this method recovers exact Bayesian inference if the model is conjugate, and empirically outperforms other online VB methods in the non-conjugate setting, such as online learning for neural networks, especially when controlling for computational costs.


Unveiling the optimization process of Physics Informed Neural Networks: How accurate and competitive can PINNs be?

Urbán, Jorge F., Stefanou, Petros, Pons, José A.

arXiv.org Artificial Intelligence

This study investigates the potential accuracy boundaries of physics-informed neural networks, contrasting their approach with previous similar works and traditional numerical methods. We find that selecting improved optimization algorithms significantly enhances the accuracy of the results. Simple modifications to the loss function may also improve precision, offering an additional avenue for enhancement. Despite optimization algorithms having a greater impact on convergence than adjustments to the loss function, practical considerations often favor tweaking the latter due to ease of implementation. On a global scale, the integration of an enhanced optimizer and a marginally adjusted loss function enables a reduction in the loss function by several orders of magnitude across diverse physical problems. Consequently, our results obtained using compact networks (typically comprising 2 or 3 layers of 20-30 neurons) achieve accuracies comparable to finite difference schemes employing thousands of grid points. This study encourages the continued advancement of PINNs and associated optimization techniques for broader applications across various fields.