Physics-informed neural networks (PINNs) have lately received great attention thanks to their flexibility in tackling a wide range of forward and inverse problems involving partial differential equations. However, despite their noticeable empirical success, little is known about how such constrained neural networks behave during their training via gradient descent. More importantly, even less is known about why such models sometimes fail to train at all. In this work, we aim to investigate these questions through the lens of the Neural Tangent Kernel (NTK); a kernel that captures the behavior of fully-connected neural networks in the infinite width limit during training via gradient descent. Specifically, we derive the NTK of PINNs and prove that, under appropriate conditions, it converges to a deterministic kernel that stays constant during training in the infinite-width limit. This allows us to analyze the training dynamics of PINNs through the lens of their limiting NTK and find a remarkable discrepancy in the convergence rate of the different loss components contributing to the total training error. To address this fundamental pathology, we propose a novel gradient descent algorithm that utilizes the eigenvalues of the NTK to adaptively calibrate the convergence rate of the total training error. Finally, we perform a series of numerical experiments to verify the correctness of our theory and the practical effectiveness of the proposed algorithms. The data and code accompanying this manuscript are publicly available at \url{https://github.com/PredictiveIntelligenceLab/PINNsNTK}.

Deep Neural Networks (DNNs) with sparse input features have been widely used in recommender systems in industry. These models have large memory requirements and need a huge amount of training data. The large model size usually entails a cost, in the range of millions of dollars, for storage and communication with the inference services. In this paper, we propose a hybrid hashing method to combine frequency hashing and double hashing techniques for model size reduction, without compromising performance. We evaluate the proposed models on two product surfaces. In both cases, experiment results demonstrated that we can reduce the model size by around 90 % while keeping the performance on par with the original baselines.

Neural networks achieve outstanding accuracy in classification and regression tasks. However, understanding their behavior still remains an open challenge that requires questions to be addressed on the robustness, explainability and reliability of predictions. We answer these questions by computing reachable sets of neural networks, i.e. sets of outputs resulting from continuous sets of inputs. We provide two efficient approaches that lead to over- and under-approximations of the reachable set. This principle is highly versatile, as we show. First, we analyze and enhance the robustness properties of both classifiers and regression models. This is in contrast to existing works, which only handle classification. Specifically, we verify (non-)robustness, propose a robust training procedure, and show that our approach outperforms adversarial attacks as well as state-of-the-art methods of verifying classifiers for non-norm bound perturbations. We also provide a technique of distinguishing between reliable and non-reliable predictions for unlabeled inputs, quantify the influence of each feature on a prediction, and compute a feature ranking.

We consider the theoretical problem of designing an optimal adversarial attack on a decision system that maximally degrades the achievable performance of the system as measured by the mutual information between the degraded signal and the label of interest. This problem is motivated by the existence of adversarial examples for machine learning classifiers. By adopting an information theoretic perspective, we seek to identify conditions under which adversarial vulnerability is unavoidable i.e. even optimally designed classifiers will be vulnerable to small adversarial perturbations. We present derivations of the optimal adversarial attacks for discrete and continuous signals of interest, i.e., finding the optimal perturbation distributions to minimize the mutual information between the degraded signal and a signal following a continuous or discrete distribution. In addition, we show that it is much harder to achieve adversarial attacks for minimizing mutual information when multiple redundant copies of the input signal are available. This provides additional support to the recently proposed ``feature compression" hypothesis as an explanation for the adversarial vulnerability of deep learning classifiers. We also report on results from computational experiments to illustrate our theoretical results.

Elon Musk has been sounding the alarm about the potentially dangerous, species-ending future of artificial intelligence for years. In 2016, the billionaire said human beings could become the equivalent of "house cats" to new AI overlords. He has since repeatedly called for regulation and caution when it comes to new AI technology. But of all the various AI projects in the works, none has Musk more worried than Google's DeepMind. "Just the nature of the AI that they're building is one that crushes all humans at all games," Musk told The New York Times in an interview.

… mention of CAs in the US Food and Drug Administration’s (FDA) proposed regulatory framework for AI or machine learning for software as a medical …

U.S. Army leaders want future armored vehicles to make decisions instantly about terrain navigation, identifying targets, and incoming enemy fire. In fact, the military wants this to happen in a matter of seconds and all without every nuance of control by humans. It is a known and often discussed concept, rapidly gaining traction as new technologies continue to emerge at rocket speed. The kinds of initiatives are now taking on a newer, more advanced character as artificial intelligence (AI)-enabled sensors, computers and targeting systems increasingly process and organize information more quickly, enabling ever-advancing measures of autonomy. Commercial applications of autonomy, such as those for driverless cars, have been advancing for quite some time, however Army developers have been taking on something quite different. Combat vehicles need autonomy not just for linear navigation but rather for an integrated series of complex, fast-changing variables such as incoming attacks, rocky terrain, air integration, and means to optimize methods of attack.

With eight successful Mars landings, NASA is upping the ante with its newest rover. The spacecraft Perseverance -- set for liftoff this week -- is NASA's biggest and brainiest Martian rover yet. It sports the latest landing tech, plus the most cameras and microphones ever assembled to capture the sights and sounds of Mars. Its super-sanitized sample return tubes -- for rocks that could hold evidence of past Martian life -- are the cleanest items ever bound for space. A helicopter is even tagging along for an otherworldly test flight.

The DARPA Subterranean (SubT) Challenge aims to develop innovative technologies that would augment operations underground. On July 20, Dr Timothy Chung, the DARPA SubTChallenge Program Manager, joined Silicon Valley Robotics to discuss the upcoming Cave Circuit and Subterranean Challenge Finals, and the opportunities that still exist for individual and team entries in both Virtual and Systems Challenges, as per the video below. The SubT Challenge allows teams to demonstrate new approaches for robotic systems to rapidly map, navigate, and search complex underground environments, including human-made tunnel systems, urban underground, and natural cave networks. The SubT Challenge is organized into two Competitions (Systems and Virtual), each with two tracks (DARPA-funded and self-funded). Teams in the Systems Competition completed four total runs, two 60-minute runs on each of two courses, Experimental and Safety Research.

Artificial Intelligence has come into everyone's life in one way or another over the past decade. One of the fastest growing areas using this new technology is the healthcare industry. There have been many advancements over the years and new ways of using it coming out every day. These advancements have even come into light with physicians using data they receive from wearables like smartwatches. Companies like Microsoft and Apple have entire teams dedicated to healthcare.