Industry
Murata beats profit estimates as AI data-center demand strains production
The company is the world's leading supplier of multilayer ceramic capacitors, essential components for every device that uses electricity because they regulate power flow. Murata Manufacturing has reported fourth-quarter earnings that beat analyst estimates, fueled by robust demand from artificial-intelligence data-center builders. Net income in the three months through March was ยฅ76.57 billion ($477 million), the Kyoto-based company said Thursday. Analysts had estimated ยฅ60 billion on average. Revenue was ยฅ460.62 billion, also better than expected.
China to ban drone sales in Beijing citing security concerns
China will ban the sale of drones in Beijing and require permits to fly them under new rules that take effect on Friday. Drones and key components will be prohibited from being sold, rented or brought into the Chinese capital. Drone owners will also be required to register their devices with the police. China has gradually tightened regulations on drones in recent years, with authorities citing public safety concerns. Drones and flying taxis are part of the so-called low-altitude economy, a strategic priority for China that is expected to generate more than two trillion yuan ($290bn; ยฃ217bn) by 2035.
Knowledge Distillation Performs Partial Variance Reduction
Knowledge distillation is a popular approach for enhancing the performance of "student" models, with lower representational capacity, by taking advantage of more powerful "teacher" models. Despite its apparent simplicity and widespread use, the underlying mechanics behind knowledge distillation (KD) are still not fully understood. In this work, we shed new light on the inner workings of this method, by examining it from an optimization perspective. We show that, in the context of linear and deep linear models, KD can be interpreted as a novel type of stochastic variance reduction mechanism. We provide a detailed convergence analysis of the resulting dynamics, which hold under standard assumptions for both strongly-convex and non-convex losses, showing that KD acts as a form of partial variance reduction, which can reduce the stochastic gradient noise, but may not eliminate it completely, depending on the properties of the "teacher" model. Our analysis puts further emphasis on the need for careful parametrization of KD, in particular w.r.t. the weighting of the distillation loss, and is validated empirically on both linear models and deep neural networks.
Streaming Algorithms and Lower Bounds for Estimating Correlation Clustering Cost
Correlation clustering is a fundamental optimization problem at the intersection of machine learning and theoretical computer science. Motivated by applications to big data processing, recent years have witnessed a flurry of results on this problem in the streaming model. In this model, the algorithm needs to process the input n-vertex graph by making one or few passes over the stream of its edges and using a limited memory, much smaller than the input size. All previous work on streaming correlation clustering has focused on semistreaming algorithms with โฆ(n) memory, whereas in this work, we study streaming algorithms with much smaller memory requirements of only polylog(n) bits. This stringent memory requirement is in the same spirit of classical streaming algorithms that instead of recovering a full solution to the problem--which can be prohibitively large with such small memory as is the case in our problem--, aimed to learn certain statistical properties of their inputs.
Static and Sequential Malicious Attacks in the Context of Selective Forgetting
With the growing demand for the right to be forgotten, there is an increasing need for machine learning models to forget sensitive data and its impact. To address this, the paradigm of selective forgetting (a.k.a machine unlearning) has been extensively studied, which aims to remove the impact of requested data from a well-trained model without retraining from scratch. Despite its significant success, limited attention has been given to the security vulnerabilities of the unlearning system concerning malicious data update requests. Motivated by this, in this paper, we explore the possibility and feasibility of malicious data update requests during the unlearning process. Specifically, we first propose a new class of malicious selective forgetting attacks, which involves a static scenario where all the malicious data update requests are provided by the adversary at once. Additionally, considering the sequential setting where the data update requests arrive sequentially, we also design a novel framework for sequential forgetting attacks, which is formulated as a stochastic optimal control problem. We also propose novel optimization algorithms that can find the effective malicious data update requests. We perform theoretical analyses for the proposed selective forgetting attacks, and extensive experimental results validate the effectiveness of our proposed selective forgetting attacks. The source code is available in the supplementary material.