[R] [1705.07832] Concrete Dropout -- learnable dropout probabilities!! • r/MachineLearning

The original one, Variational Dropout and the Local Reparameterization Trick is cited in the Concrete Dropout paper and is indeed somewhat limited, however this issue is resolved in Variational Dropout Sparsifies Deep Neural Networks (accepted to ICML '17, paper from my labmates). They have very strange excuse to avoid comparison with the last paper (IMO both methods use different relaxations, it'd be useful to compare them face-to-face) We chose not to compare to Gaussian dropout in our experiments, as when optimising Gaussian dropout's α following its variational interpretation [23], the method is known to underperform [28] UPD: there's also Generalized Dropout (uses straight through estimator, which is not unbiased gradient estimator, and Information Dropout that does not use binary formulation.