Government
On the Importance of Calibration in Semi-supervised Learning
Loh, Charlotte, Dangovski, Rumen, Sudalairaj, Shivchander, Han, Seungwook, Han, Ligong, Karlinsky, Leonid, Soljacic, Marin, Srivastava, Akash
State-of-the-art (SOTA) semi-supervised learning (SSL) methods have been highly successful in leveraging a mix of labeled and unlabeled data by combining techniques of consistency regularization and pseudo-labeling. During pseudo-labeling, the model's predictions on unlabeled data are used for training and thus, model calibration is important in mitigating confirmation bias. Yet, many SOTA methods are optimized for model performance, with little focus directed to improve model calibration. In this work, we empirically demonstrate that model calibration is strongly correlated with model performance and propose to improve calibration via approximate Bayesian techniques. We introduce a family of new SSL models that optimizes for calibration and demonstrate their effectiveness across standard vision benchmarks of CIFAR-10, CIFAR-100 and ImageNet, giving up to 15.9% improvement in test accuracy. Furthermore, we also demonstrate their effectiveness in additional realistic and challenging problems, such as class-imbalanced datasets and in photonics science.
Hierarchical Learning in Euclidean Neural Networks
Rackers, Joshua A., Rao, Pranav
Equivariant machine learning methods have shown wide success at 3D learning applications in recent years. These models explicitly build in the reflection, translation and rotation symmetries of Euclidean space and have facilitated large advances in accuracy and data efficiency for a range of applications in the physical sciences. An outstanding question for equivariant models is why they achieve such larger-than-expected advances in these applications. To probe this question, we examine the role of higher order (non-scalar) features in Euclidean Neural Networks (\texttt{e3nn}). We focus on the previously studied application of \texttt{e3nn} to the problem of electron density prediction, which allows for a variety of non-scalar outputs, and examine whether the nature of the output (scalar $l=0$, vector $l=1$, or higher order $l>1$) is relevant to the effectiveness of non-scalar hidden features in the network. Further, we examine the behavior of non-scalar features throughout training, finding a natural hierarchy of features by $l$, reminiscent of a multipole expansion. We aim for our work to ultimately inform design principles and choices of domain applications for {\tt e3nn} networks.
A Reduction to Binary Approach for Debiasing Multiclass Datasets
Alabdulmohsin, Ibrahim, Schrouff, Jessica, Koyejo, Oluwasanmi
We propose a novel reduction-to-binary (R2B) approach that enforces demographic parity for multiclass classification with non-binary sensitive attributes via a reduction to a sequence of binary debiasing tasks. We prove that R2B satisfies optimality and bias guarantees and demonstrate empirically that it can lead to an improvement over two baselines: (1) treating multiclass problems as multi-label by debiasing labels independently and (2) transforming the features instead of the labels. Surprisingly, we also demonstrate that independent label debiasing yields competitive results in most (but not all) settings.
Diversity-aware social robots meet people: beyond context-aware embodied AI
Recchiuto, Carmine, Sgorbissa, Antonio
Carmine Recchiuto, Antonio Sgorbissa Introduction Mayra is a 34-year-old woman from Sri Lanka who arrived in Genoa in 2020, just before the COVID-19 pandemic. Mayra spends her days taking care of her three children and doing housework. Due to the lockdown measures, she had few opportunities to develop relationships with Italian people, so her Italian has remained very basic. The situation did not improve until December 2021 because finding a job was challenging due to the remaining COVID restriction. In January 2022, her husband bought a small robot, and Mayra called it "Dhvija."
A Quantitative Geometric Approach to Neural-Network Smoothness
Wang, Zi, Prakriya, Gautam, Jha, Somesh
Fast and precise Lipschitz constant estimation of neural networks is an important task for deep learning. Researchers have recently found an intrinsic trade-off between the accuracy and smoothness of neural networks, so training a network with a loose Lipschitz constant estimation imposes a strong regularization and can hurt the model accuracy significantly. In this work, we provide a unified theoretical framework, a quantitative geometric approach, to address the Lipschitz constant estimation. By adopting this framework, we can immediately obtain several theoretical results, including the computational hardness of Lipschitz constant estimation and its approximability. Furthermore, the quantitative geometric perspective can also provide some insights into recent empirical observations that techniques for one norm do not usually transfer to another one. We also implement the algorithms induced from this quantitative geometric approach in a tool GeoLIP. These algorithms are based on semidefinite programming (SDP). Our empirical evaluation demonstrates that GeoLIP is more scalable and precise than existing tools on Lipschitz constant estimation for $\ell_\infty$-perturbations. Furthermore, we also show its intricate relations with other recent SDP-based techniques, both theoretically and empirically. We believe that this unified quantitative geometric perspective can bring new insights and theoretical tools to the investigation of neural-network smoothness and robustness.
Instance-Based Uncertainty Estimation for Gradient-Boosted Regression Trees
Brophy, Jonathan, Lowd, Daniel
Gradient-boosted regression trees (GBRTs) are hugely popular for solving tabular regression problems, but provide no estimate of uncertainty. We propose Instance-Based Uncertainty estimation for Gradient-boosted regression trees (IBUG), a simple method for extending any GBRT point predictor to produce probabilistic predictions. IBUG computes a non-parametric distribution around a prediction using the $k$-nearest training instances, where distance is measured with a tree-ensemble kernel. The runtime of IBUG depends on the number of training examples at each leaf in the ensemble, and can be improved by sampling trees or training instances. Empirically, we find that IBUG achieves similar or better performance than the previous state-of-the-art across 22 benchmark regression datasets. We also find that IBUG can achieve improved probabilistic performance by using different base GBRT models, and can more flexibly model the posterior distribution of a prediction than competing methods. We also find that previous methods suffer from poor probabilistic calibration on some datasets, which can be mitigated using a scalar factor tuned on the validation data. Source code is available at https://www.github.com/jjbrophy47/ibug.
Lab explores using AI helping cops catch criminals – EFF
In brief America's Pacific Northwest National Laboratory is looking into how AI technologies can be used to create a "Digital Police Officer" or "D-PO" in the future. Freedom-of-information requests filed by the Electronic Frontier Foundation show the US Department of Energy-funded lab envisions cops may one day be able to partner up with a virtual crime-fighting assistant. D-PO would be capable of, for instance, tapping into facial recognition systems to alert a police officer on patrol to a suspect nearby, and can even offer advice on how best to apprehend the suspect. The EFF warned against the plod teaming up with software like D-PO, citing concerns over inaccurate facial recognition matches and biased predictive policing policies. "The good news is that in the emails we obtained, one of the authors acknowledges in internal emails that elements like a D-PO taking over driving is a'long way off' and monitoring live drone feeds is'not a near-term capability,' the digital privacy-focused non-profit said. The national lab has also described how a separate virtual assistant, BITS, could provide US border and customs agents with visual data to help them crack down on narcotics trafficking. "The records EFF received do not indicate any official interest from CBP or the Department of Homeland Security.
White House Office of Science and Technology Policy Releases AI Bill of Rights
This morning, the White House Office of Science and Technology Policy released a long-awaited "Blueprint for an AI Bill of Rights" ("AI Bill of Rights") that, when implemented, would apply to automated systems that have the potential to meaningfully affect the American public's rights, opportunities, or access to critical resources or services. The AI Bill of Rights is designed to provide protections to apply broadly to all automated systems that "have the potential" to significantly affect individuals or communities, from civil rights/civil liberties (including privacy), to equal opportunities for healthcare, education, and employment, as well as access to resources and services. The AI Bill of Rights contains five broad categories of practices designed to "guide the design, use, and deployment of automated systems to protect the rights of the American public in the age of artificial intelligence." Safe and Effective Systems: Individuals "should be protected from unsafe or ineffective systems." In addition, "[a]utomated systems should be developed with consultation from diverse communities, stakeholders, and domain experts to identify concerns, risks, and potential impacts of the system."
The White House unveils a "Bill of Rights" for artificial intelligence
As artificial intelligence continues to develop and become a bigger part of our lives, many people believe AI should have regulations or guidelines in place. The White House agrees with those citizens, releasing a "bill of rights" dedicated solely to artificial intelligence. It would be a massive understatement to say artificial intelligence has affected our lives. Whether that's a positive or negative is entirely up to you. Virtual assistants like Alexa and Siri are integrated into many devices that people interact with everyday.
Robotics
A "robot" is a machine that's designed by humans to do a specific job. And the scientists who design and build robots are called "roboticists". Robots do job that people can't do or don't want to do. Like if the job is boring, if it involves doing the same thing over and over or if a job is very dangerous and it means going places where people could get hurt, then robots are used, to do that job. Basically, robot is any automatically operated machine that replaces human effort, though it may resemble humans in appearance or perform functions like humans.