Normalization layers (e.g., Batch Normalization, Layer Normalization) were introduced to help with optimization difficulties in very deep nets, but they clearly also

In this paper, we tackle the important yet under-investigated problem of making long-horizon prediction of event sequences. Existing state-of-the-art models do not perform well at this task due to their autoregressive structure.

"We were very much impressed. At the same time, we were afraid." Inside the quest to map the universe with mysterious bursts of radio energy When our universe was less than half as old as it is today, a burst of energy that could cook a sun's worth of popcorn shot out from somewhere amid a compact group of galaxies. Some 8 billion years later, radio waves from that burst reached Earth and were captured by a sophisticated low-frequency radio telescope in the Australian outback. The signal, which arrived in June 2022, and lasted for under half a millisecond, is one of a growing class of mysterious radio signals called fast radio bursts. In the last 10 years, astronomers have picked up nearly 5,000 of them.

Here the DAE dramatically simplifies the search space by mapping inputs into a continuous latent space where familiar Bayesian optimization tools can be more readily applied. Despite this simplification, the latent space typically remains high-dimensional.

Deep learning methods have made large strides in building task-specific models, but are shown to easily forget past knowledge when learning new tasks [4, 5].

For instance, the existence of spurious local minima may not pose any issue if an optimization algorithm can avoid them efficiently.