Statistical Learning
ErrorCompensatedX: error compensation for variance reduced algorithms
Communication cost is one major bottleneck for the scalability for distributed learning.
Dynamic Bottleneck for Robust Self-Supervised Exploration
However, such methods are usually sensitive to environmental dynamics-irrelevant information, e.g., white-noise. To handle such dynamics-irrelevant information, we propose a Dynamic Bottleneck (DB) model, which attains a dynamics-relevant representation based on the information-bottleneck principle.
On skip connections and normalisation layers in deep optimisation
We then demonstrate the utility of this framework in two respects.
Self-Supervised Visual Representation Learning with Semantic Grouping
In this paper, we tackle the problem of learning visual representations from unlabeled scene-centric data. Existing works have demonstrated the potential of utilizing the underlying complex structure within scene-centric data; still, they commonly rely on hand-crafted objectness priors or specialized pretext tasks to build a learning framework, which may harm generalizability. Instead, we propose contrastive learning from data-driven semantic slots, namely SlotCon, for joint semantic grouping and representation learning.
An Exponential Improvement on the Memorization Capacity of Deep Threshold Networks
It is well known that modern deep neural networks are powerful enough to memorize datasets even when the labels have been randomized.