The Gumbel-Max trick is the basis of many relaxed gradient estimators. These estimators areeasy toimplement and lowvariance, butthegoal ofscaling them comprehensively to large combinatorial distributions is still outstanding. Working within the perturbation model framework, we introduce stochastic softmax tricks, which generalizetheGumbel-Softmax tricktocombinatorial spaces.

We consider the problem of maximizing submodular functions in single-pass streaming and secretaries-with-shortlists models, both with random arrival order. For cardinality constrained monotone functions, Agrawal, Shadravan, and Stein [ASS19]gaveasingle-pass(1 1/e ε)-approximation algorithm using only linear memory,buttheir exponential dependence onεmakesitimpractical evenforε = 0.1.

Although the fairness community has recognized the importance of data, re-searchers inthe area primarily rely on UCIAdult when itcomes totabular data.

However, neither approach has achieved the anticipated results.

The detailed dimensions are showninTable2.

Weshowthattheanswerisno,intworespects: First, we show that hard positives are being neglected in the self-supervised training, and that if these hard positives are included then the quality of learnt representation improves significantly. Toinvestigatethis,weconduct anoracleexperiment where positivesamples areincorporated into the instance-based training process based on the semantic class label.