Country
Sparse Symplectically Integrated Neural Networks
While impressive, most of these tasks can be easily performed by an intelligent human.
Does Knowledge Distillation Really Work? Samuel Stanton NYU Pavel Izmailov NYU Polina Kirichenko NYU Alexander A. Alemi Google Research Andrew Gordon Wilson NYU
Does Knowledge Distillation Really Work? Are We Usingthe Right Distillation Dataset?
439d8c975f26e5005dcdbf41b0d84161-Paper.pdf
We further give "active local" versions of these heuristics: given atest pointx?,we show how the labelT(x?) With this information, we may decide thathwould not have been of much utility anyway, thereby saving ourselves the resources and effort to label the entire datasetS (and to runA).
Supplemental Material: Meta-learning from Tasks with Heterogeneous Attribute Spaces
With NP, we used deep sets for handling tasks with heterogeneous attribute spaces. DS+FT (NP+FT) was the DS (NP) fine-tuned with each target dataset. The number of fine-tuning epochs was five. NP+FT, NP+MAML, and the proposed method. Results Table 2 shows the mean squared error for each target task.