Technology
A Single-Step, Sharpness-Aware Minimization is All You Need to Achieve Efficient and Accurate Sparse Training
However, the training of a sparse DNN encounters great challenges in achieving optimal generalization ability despite the efforts from the state-of-the-art sparse training methodologies. To unravel the mysterious reason behind the difficulty of sparse training, we connect network sparsity with the structure of neural loss functions and identify that the cause of such difficulty lies in a chaotic loss surface.
Monte-Carlo Tree Search for Constrained POMDPs
Jongmin Lee, Geon-hyeong Kim, Pascal Poupart, Kee-Eung Kim
Neural Information Processing Systems http://nips.cc/
From Complexity to Simplicity: Adaptive ES-Active Subspaces for Blackbox Optimization
Krzysztof M. Choromanski, Aldo Pacchiano, Jack Parker-Holder, Yunhao Tang, Vikas Sindhwani
Neural Information Processing Systems http://nips.cc/
UniXGrad: A Universal, Adaptive Algorithm with Optimal Guarantees for Constrained Optimization
Ali Kavis, Kfir Y. Levy, Francis Bach, Volkan Cevher
Stochastic constrained optimization with first-order oracles (SCO) is critical in machine learning.
Generative Probabilistic Novelty Detection with Adversarial Autoencoders
Stanislav Pidhorskyi, Ranya Almohsen, Gianfranco Doretto
We assume that training data is available to describe only the inlier distribution. Recent approaches primarily leverage deep encoder-decoder network architectures to compute a reconstruction error that is used to either compute a novelty score or to train a one-class classifier.
SingularValueFine-tuning: Few-shotSegmentation requiresFew-parametersFine-tuning-SupplementaryMaterial
Different finetune strategy: In Figure 1, we visualize the mIoU curve of different fine-tuning strategies. It can be seen that both layer-based and convolution-based fine-tuning methods bring over-fitting problems. This result shows that traditional fine-tuning methods are not suitable for few-shot segmentation tasks. Directly fine-tuning theparameters ofbackbone infew-shot learning affects the robustness ofFSS models. Therefore, we propose anovelfine-tuning strategy,namely SVF.
