A central challenge in training classification models in the real-world federated system is learning with non-IID data. To cope with this, most of the existing works involve enforcing regularization in local optimization or improving the model aggregation scheme at the server. Other works also share public datasets or synthesized samples to supplement the training of under-represented classes or introduce a certain level of personalization. Though effective, they lack a deep understanding of how the data heterogeneity affects each layer of a deep classification model. In this paper, we bridge this gap by performing an experimental analysis of the representations learned by different layers. Our observations are surprising: (1) there exists a greater bias in the classifier than other layers, and (2) the classification performance can be significantly improved by post-calibrating the classifier after federated training. Motivated by the above findings, we propose a novel and simple algorithm called Classifier Calibration with Virtual Representations (CCVR), which adjusts the classifier using virtual representations sampled from an approximated gaussian mixture model. Experimental results demonstrate that CCVR achieves state-of-the-art performance on popular federated learning benchmarks including CIFAR-10, CIFAR-100, and CINIC-10. We hope that our simple yet effective method can shed some light on the future research of federated learning with non-IID data.

This paper re-examines a continuous optimization framework dubbed NOTEARS for learning Bayesian networks. We first generalize existing algebraic characterizations of acyclicity to a class of matrix polynomials. Next, focusing on a one-parameter-per-edge setting, it is shown that the Karush-Kuhn-Tucker (KKT) optimality conditions for the NOTEARS formulation cannot be satisfied except in a trivial case, which explains a behavior of the associated algorithm. We then derive the KKT conditions for an equivalent reformulation, show that they are indeed necessary, and relate them to explicit constraints that certain edges be absent from the graph. If the score function is convex, these KKT conditions are also sufficient for local minimality despite the non-convexity of the constraint. Informed by the KKT conditions, a local search post-processing algorithm is proposed and shown to substantially and universally improve the structural Hamming distance of all tested algorithms, typically by a factor of 2 or more. Some combinations with local search are both more accurate and more efficient than the original NOTEARS.

'It's going to be really bad': Fears over AI bubble bursting grow in Silicon Valley At OpenAI's DevDay this week, OpenAI boss Sam Altman did what American tech bosses rarely do these days: he actually answered questions from reporters. I know it's tempting to write the bubble story, Mr Altman told me as he sat flanked by his top lieutenants. In fact, there are many parts of AI that I think are kind of bubbly right now. In Silicon Valley, the debate over whether AI companies are overvalued has taken on a new urgency. Sceptics are privately - and some now publicly - asking whether the rapid rise in the value of AI tech companies may be, at least in part, the result of what they call financial engineering.

The highly contentious and arguably irresponsible comments from Alibaba founder Jack Ma around AI and its likelihood of creating a third World War – will have done little to inspire confidence in those that harbour fears around the subject of intelligent machines. For some, the two words placed together spark a sense of dread, trepidation or even fear. For others, it represents the beginning of an exciting new digital world with untold benefits and opportunities. Unfortunately, however, it's often the former, which seems to seep more into people's consciousness. It's perhaps then of little surprise that in a recent survey by the British Science Association (BSA) that 36% of respondents believe that AI will eventually takeover or destroy humanity.

Facebook has rolled out a new technology called "proactive detection" artificial intelligence (AI) that would scan all of a user's posts seeking patterns that suggest suicidal thoughts. Also, if found necessary, it would share helpful information to either the user or his/her friends, or it may get in touch with local first-responders. Responding to a question by website TechCrunch, a Facebook spokesperson said that users cannot opt out of the feature. The spokesperson further said that the new feature is meant to ensure better user safety and the support resources that the social media behemoth is offering could be easily dismissed if the user doesn't wish to check them. Since an AI is expected to sense suicidal patterns faster, Facebook, through its deployment, is hoping to bring down the time taken to provide help to those at risk.

Elon Musk came out with another artificial intelligence warning on Monday.