It also stores approximate per-layer copies of activations, at significantmemorysavings,thatareusedinthebackwardpass.

Training convolutional neural network models is memory intensive since back-propagation requires storing activations of all intermediate layers.

Overall summary of the paper: This paper proposed a multi-task learning algorithm from multi-objective optimization perspective and the authors provided an approximation algorithm, which could accelerate the training process. The authors claim that existing MTL algorithms used linear combinations (uniform weight) of the loss from different tasks, which is hard to achieve the Pareto optimality. Unlike the uniform weight strategy, the authors use the MGDA algorithm to solve the optimal weight, which would increase the performance for all the tasks to achieve the Pareto optimality. Moreover, when solving the sub-problem for shared parameters, the author gave an upper bound of the loss function, and this upper bound optimization only requires one-time back propagation regardless of the number of tasks. The results show that the approximation strategy not only can accelerate the training process but also improve the performance.

Machine learning (ML) models that achieve high average accuracy can still underperform on semantically coherent subsets (i.e. "slices") of data. This behavior can have significant societal consequences for the safety or bias of the model in deployment, but identifying these underperforming slices can be difficult in practice, especially in domains where practitioners lack access to group annotations to define coherent subsets of their data. Motivated by these challenges, ML researchers have developed new slice discovery algorithms that aim to group together coherent and high-error subsets of data. However, there has been little evaluation focused on whether these tools help humans form correct hypotheses about where (for which groups) their model underperforms. We conduct a controlled user study (N = 15) where we show 40 slices output by two state-of-the-art slice discovery algorithms to users, and ask them to form hypotheses about where an object detection model underperforms. Our results provide positive evidence that these tools provide some benefit over a naive baseline, and also shed light on challenges faced by users during the hypothesis formation step. We conclude by discussing design opportunities for ML and HCI researchers. Our findings point to the importance of centering users when designing and evaluating new tools for slice discovery.

Incomplete MaxSAT solving aims to quickly find a solution that attempts to minimize the sum of the weights of the unsatisfied soft clauses without providing any optimality guarantees. In this paper, we propose two approximation strategies for improving incomplete MaxSAT solving. In one of the strategies, we cluster the weights and approximate them with a representative weight. In another strategy, we break up the problem of minimizing the sum of weights of unsatisfiable clauses into multiple minimization subproblems. Experimental results show that approximation strategies can be used to find better solutions than the best incomplete solvers in the MaxSAT Evaluation 2017.