Education
Supplementary Material for Paper " Terra: Imperative-Symbolic Co-Execution of Imperative Deep Learning Programs "
For example, the MatMul operation of TensorFlow has'MatMul' as As same as the call id stack, Terra manages the loop id stack for the entire program execution. Figure 2: The result of the case assignment algorithm for the given TraceGraph.2 4 In this section, we describe the case assignment algorithm that Terra uses to explicitly insert the Switch-Case operations in the symbolic graph. The algorithm takes a TraceGraph as an input and returns an ordered list of switch-cases. A switch-case 6is a set of (basic block, control edges) where thebasic block is a linear3 chain of nodes, and the5control edges are the edges that point to the basic block. Every non-overlapping linear chain of nodes in the TraceGraph is uniquely assigned to a basic block so that the ordered list of3switch-cases 5can cover every trace in the TraceGraph.
Optimality and Stability in Federated Learning: AGame-theoretic Approach
Federated learning is a distributed learning paradigm where multiple agents, each only with access to local data, jointly learn a global model. There has recently been an explosion of research aiming not only to improve the accuracy rates of federated learning, but also provide certain guarantees around social good properties such as total error. One branch of this research has taken a game-theoretic approach, and in particular, prior work has viewed federated learning as a hedonic game, where error-minimizing players arrange themselves into federating coalitions. This past work proves the existence of stable coalition partitions, but leaves open a wide range of questions, including how far from optimal these stable solutions are. In this work, we motivate and define a notion of optimality given by the average error rates among federating agents (players).
Distilling Image Classifiers in Object Detectors
Knowledge distillation constitutes a simple yet effective way to improve the performance of a compact student network by exploiting the knowledge of a more powerful teacher. Nevertheless, the knowledge distillation literature remains limited to the scenario where the student and the teacher tackle the same task. Here, we investigate the problem of transferring knowledge not only across architectures but also across tasks. To this end, we study the case of object detection and, instead of following the standard detector-to-detector distillation approach, introduce a classifier-to-detector knowledge transfer framework. In particular, we propose strategies to exploit the classification teacher to improve both the detector's recognition accuracy and localization performance. Our experiments on several detectors with different backbones demonstrate the effectiveness of our approach, allowing us to outperform the state-of-the-art detector-to-detector distillation methods.