Forinstance, one candefine anewupper bounding5 function that returns the minimum of two other upper bounding functions. The difficulty is in finding other upper6 bounding functions that are (1) tighter than the bound we use, (2) efficient to compute, and (3) empirically require7 few calls toRefine. Unfortunately,thefastcimplementation ofthe11 bound provided by [1] is numerically unstable for sparse matrices with 0 entries and the numerically stable matlab12 implementation is prohibitively slow. This difficulty could be overcome by rewriting an efficient implementation.13 This14 bound is computed by solving an optimization problem, but unfortunately we do not know of an efficient solution.15

Neural Information Processing Systems http://nips.cc/

Toeasethelearning, wedemonstrate thatitisbeneficial toadoptacurriculum learning strategy [23], where harder negatives are introduced after an initial stage of learning on easiernegatives.

We also introduce a new Gated Recursive Cell to compose lower levelrepresentations into higher level representation. We demonstrate that our modelachieves strong performance on the logical inference task (Bowman et al., 2015)andtheListOps(NangiaandBowman,2018)task. Wecanalsointerpretthemodelto retrieve the induced tree structure, and find that these induced structures alignwith the ground truth.

The purpose ofthis is to scale down the logits before softmax is applied, a technique similar to the one seen in Vaswani et al. (2017). We will correct the typos in the paper for the camera-ready version. We understand your criticism with respect to ablation studies. Attention is all you need.

Using these smoothed rank andsort operators, we propose differentiable proxies for the classification 0/1 loss as well as for the quantileregressionloss.

In an attempt to reduce their description size, we study low rank approximation of such matrices.

For details on M1-M5, please refer to Appendix B.3.