ShareThis week, we will briefly touch upon the topic of artificial intelligence, or AI, discuss what it is, why it is so important to the American empire and national security, and how the ancient Chamorro people of the Marianas can prepare themselves to be ready for future possible job and entrepreneurial opportunities at the intersection of warfare and technology.  What is AI? Star Wars gone wild?  AI is a constellation or universe of technologies, computer hardware and software, designed to solve specific tasks that reflect and are intended to resemble human cognitive processes to include decision making, reasoning, learning, and perceiving.  AI has applications that reprogram itself to complete specific tasks called machine learning. Within the machine learning space, there are applications that take information to produce outcomes with increasing accuracy. This technology can be interpreted as an evolving manufactured ecosystem that attempts to some degree to self-develop and monitor without human intervention.  This is both potentially dangerous and beneficial stuff.  Why AI is important to the United States and our ancient Chamorro people Part of the answer is found in a report released by the congressionally established National Security Commission on Artificial Intelligence, which outlined the national importance of AI and its application to all facets of American—and by implication, American colonial—society. The commission was established to assess how AI affects imperial competitiveness and technological advantage.  AI is important because it has civilian and military applications that are used every day by every citizen, whether one resides on Saipan, Rota, in New York, Birmingham or elsewhere. If you are looking for a movie to watch or check out what time a restaurant or bar opens, or get vehicle maintenance service, or asking your iPhone a question, AI will be a part of the enabling infrastructure that will get you the answers to your questions.  AI is now found everywhere and is used knowingly or unknowingly, again posing benefits and concerns to society.  If Guam moves toward building a new hospital, AI will be an integral and ubiquitous part of how healthcare decisions are informed, how biotechnology is implemented, and how personal medical information and data are managed, distributed, protected, and delivered. AI will become more important as Guam continues to move toward technological solutions for future energy, food technology and security opportunities. Folks in the NMI will see a greater role of AI as the Commonwealth moves toward electric and eventually unmanned cars. Cyber-attacks against the United States government and other facets of society occur every day. AI was the underbelly used to enable computer networks to find vulnerabilities recently when the government of Guam experienced a cyber-attack. These attacks may come in the form of data harvesting, targeted attacks on individual citizens, or AI enabled attacks on social media, intended to influence or harm the targets.   The American national government is contemplating ways to aggressively deal with data protection, privacy, and security. Congress remains behind the curve on how to craft legislation to address ongoing technological change. Others worry that domestic data surveillance is out of control and has already compromised our privacy and protection.  Why AI is important to the governments of Guam and the CNMI Now is the time for the governments of Guam and the CNMI to consider creating a Marianas Artificial Intelligence, Security and Emerging Technologies Understanding advisory board to learn and more completely seek to comprehend the nature of AI, how it is currently used and how it presents opportunities and vulnerabilities to every aspect of Pacific island life. There is no reason why Guam and CNMI legislative committees overseeing technology cannot hold initial hearings to discuss AI.  Future educational and career opportunities: Pay attention Guam Community College is doing some work with its resources available to students in math and science tutoring and management information systems programming. The University of Guam has resources that can or will eventually be able to contribute toward providing AI jobs for young islanders who have math, engineering, nursing, education, biology, and Chamorro studies backgrounds.  Guam institutions have opportunities to further partner with the Guam Department of Education, and the CNMI Public School System on the implications of AI. Marianas educational institutions would be well served to contemplate the establishment of emerging technology certificate programs to prepare Pacific Islanders for much needed and well-paid technology jobs. All Marianas institutions can also create opportunities to seek partnerships with major American companies and entities in Silicon Valley and elsewhere, intended to create AI related jobs and training.  Lawmakers from Guam and the CNMI as well as the governors can contemplate creating national level digital annex risk management programs for all school age kids interested in future technical challenges that will provide good paying jobs while helping to secure America and its colonies. Science and technology partnerships with Taiwan and South Korea may also be something that can be operationalized for mutual benefit.  The AI race is on, security threats increase A most dangerous aspect of rapidly emerging AI enabled technology and networks is that nation state adversaries such as China and Russia may outpace the United States on this front over the next 10 to 15 years. This presents and will present fundamental risks and threats to the existing militarized resource base currently embedded and/or connected to Guam and the CNMI that may include risks tied to autonomous unmanned weapon system warfare and real future risks associated with crisis stability and human authorized employment of very dangerous nuclear weapons and networks through unmanned platforms.  AI threatens to pose real challenges to traditional process identification and validation issues related to military targeting matters, creating risks to real-time battle assessment decisions that will need to be made in the future.   Now is the time for our Chamorro Pacific Islander civilization to continue the broad conversation on issues of the military and national and colonial security to demystify and assess the importance of AI and what it means to all our families and friends in the 21st century. 

When a secretive start-up scraped the internet to build a facial-recognition tool, it tested a legal and ethical limit -- and blew the future of privacy in America wide open. In May 2019, an agent at the Department of Homeland Security received a trove of unsettling images. Found by Yahoo in a Syrian user's account, the photos seemed to document the sexual abuse of a young girl. One showed a man with his head reclined on a pillow, gazing directly at the camera. The man appeared to be white, with brown hair and a goatee, but it was hard to really make him out; the photo was grainy, the angle a bit oblique. The agent sent the man's face to child-crime investigators around the country in the hope that someone might recognize him. When an investigator in New York saw the request, she ran the face through an unusual new facial-recognition app she had just started using, called Clearview AI. The team behind it had scraped the public web -- social media, employment sites, YouTube, Venmo -- to create a database with three billion images of people, along with links to the webpages from which the photos had come. This dwarfed the databases of other such products for law enforcement, which drew only on official photography like mug shots, driver's licenses and passport pictures; with Clearview, it was effortless to go from a face to a Facebook account. The app turned up an odd hit: an Instagram photo of a heavily muscled Asian man and a female fitness model, posing on a red carpet at a bodybuilding expo in Las Vegas. The suspect was neither Asian nor a woman. But upon closer inspection, you could see a white man in the background, at the edge of the photo's frame, standing behind the counter of a booth for a workout-supplements company. On Instagram, his face would appear about half as big as your fingernail. The federal agent was astounded. The agent contacted the supplements company and obtained the booth worker's name: Andres Rafael Viola, who turned out to be an Argentine citizen living in Las Vegas.

We propose LSDAT, an image-agnostic decision-based black-box attack that exploits low-rank and sparse decomposition (LSD) to dramatically reduce the number of queries and achieve superior fooling rates compared to the state-of-the-art decision-based methods under given imperceptibility constraints. LSDAT crafts perturbations in the low-dimensional subspace formed by the sparse component of the input sample and that of an adversarial sample to obtain query-efficiency. The specific perturbation of interest is obtained by traversing the path between the input and adversarial sparse components. It is set forth that the proposed sparse perturbation is the most aligned sparse perturbation with the shortest path from the input sample to the decision boundary for some initial adversarial sample (the best sparse approximation of shortest path, likely to fool the model). Theoretical analyses are provided to justify the functionality of LSDAT. Unlike other dimensionality reduction based techniques aimed at improving query efficiency (e.g, ones based on FFT), LSD works directly in the image pixel domain to guarantee that non-$\ell_2$ constraints, such as sparsity, are satisfied. LSD offers better control over the number of queries and provides computational efficiency as it performs sparse decomposition of the input and adversarial images only once to generate all queries. We demonstrate $\ell_0$, $\ell_2$ and $\ell_\infty$ bounded attacks with LSDAT to evince its efficiency compared to baseline decision-based attacks in diverse low-query budget scenarios as outlined in the experiments.

We describe a Plug-and-Play controllable language generation framework, Plug-and-Blend, that allows a human user to input multiple control codes (topics). In the context of automated story generation, this allows a human user lose or fine grained control of the topics that will appear in the generated story, and can even allow for overlapping, blended topics. We show that our framework, working with different generation models, controls the generation towards given continuous-weighted control codes while keeping the generated sentences fluent, demonstrating strong blending capability.

Reducing the variance of the gradient estimator is known to improve the convergence rate of stochastic gradient-based optimization and sampling algorithms. One way of achieving variance reduction is to design importance sampling strategies. Recently, the problem of designing such schemes was formulated as an online learning problem with bandit feedback, and algorithms with sub-linear static regret were designed. In this work, we build on this framework and propose Avare, a simple and efficient algorithm for adaptive importance sampling for finite-sum optimization and sampling with decreasing step-sizes. Under standard technical conditions, we show that Avare achieves $\mathcal{O}(T^{2/3})$ and $\mathcal{O}(T^{5/6})$ dynamic regret for SGD and SGLD respectively when run with $\mathcal{O}(1/t)$ step sizes. We achieve this dynamic regret bound by leveraging our knowledge of the dynamics defined by the algorithm, and combining ideas from online learning and variance-reduced stochastic optimization. We validate empirically the performance of our algorithm and identify settings in which it leads to significant improvements.

Semi-supervised learning has recently emerged as one of the most promising paradigms to mitigate the reliance of deep learning on huge amounts of labeled data, especially in learning tasks where it is costly to collect annotated data. This is best illustrated in medicine, where measurement require overpriced machinery and labels are the result of an expensive human assisted time-consuming analysis. Semi-supervised learning (SSL) aims to largely reduce the need for massive labeled datasets by allowing a model to leverage both labeled and unlabeled data. Among the many semi-supervised learning approaches, graph-based semi-supervised learning techniques are increasingly being studied due to their performance and to more and more real graph datasets. The problem is to predict all the unlabelled vertices in the graph based on only a small subset of vertices being observed. To date, a number of graph-based algorithms, in particular label propagation methods have been successfully applied to different fields, such as social network analysis [7][50][51][25], natural language processing [1][43][3], and image segmentation [47][10]. The performance of label propagation algorithms is often affected by the graph-construction method and the technique of inferring pseudo-labels.

We established a Spatio-Temporal Neural Network, namely STNN, to forecast the spread of the coronavirus COVID-19 outbreak worldwide in 2020. The basic structure of STNN is similar to the Recurrent Neural Network (RNN) incorporating with not only temporal data but also spatial features. Two improved STNN architectures, namely the STNN with Augmented Spatial States (STNN-A) and the STNN with Input Gate (STNN-I), are proposed, which ensure more predictability and flexibility. STNN and its variants can be trained using Stochastic Gradient Descent (SGD) algorithm and its improved variants (e.g., Adam, AdaGrad and RMSProp). Our STNN models are compared with several classical epidemic prediction models, including the fully-connected neural network (BPNN), and the recurrent neural network (RNN), the classical curve fitting models, as well as the SEIR dynamical system model. Numerical simulations demonstrate that STNN models outperform many others by providing more accurate fitting and prediction, and by handling both spatial and temporal data.

A vastly growing literature on explaining deep learning models has emerged. This paper contributes to that literature by introducing a global gradient-based model-agnostic method, which we call Marginal Attribution by Conditioning on Quantiles (MACQ). Our approach is based on analyzing the marginal attribution of predictions (outputs) to individual features (inputs). Specificalllly, we consider variable importance by mixing (global) output levels and, thus, explain how features marginally contribute across different regions of the prediction space. Hence, MACQ can be seen as a marginal attribution counterpart to approaches such as accumulated local effects (ALE), which study the sensitivities of outputs by perturbing inputs. Furthermore, MACQ allows us to separate marginal attribution of individual features from interaction effect, and visually illustrate the 3-way relationship between marginal attribution, output level, and feature value.

In recent years, deep learning techniques have outperformed traditional models in many machine learning tasks. Deep neural networks have successfully been applied to address time series forecasting problems, which is a very important topic in data mining. They have proved to be an effective solution given their capacity to automatically learn the temporal dependencies present in time series. However, selecting the most convenient type of deep neural network and its parametrization is a complex task that requires considerable expertise. Therefore, there is a need for deeper studies on the suitability of all existing architectures for different forecasting tasks. In this work, we face two main challenges: a comprehensive review of the latest works using deep learning for time series forecasting; and an experimental study comparing the performance of the most popular architectures. The comparison involves a thorough analysis of seven types of deep learning models in terms of accuracy and efficiency. We evaluate the rankings and distribution of results obtained with the proposed models under many different architecture configurations and training hyperparameters. The datasets used comprise more than 50000 time series divided into 12 different forecasting problems. By training more than 38000 models on these data, we provide the most extensive deep learning study for time series forecasting. Among all studied models, the results show that long short-term memory (LSTM) and convolutional networks (CNN) are the best alternatives, with LSTMs obtaining the most accurate forecasts. CNNs achieve comparable performance with less variability of results under different parameter configurations, while also being more efficient.

With the success of large-scale pre-training and multilingual modeling in Natural Language Processing (NLP), recent years have seen a proliferation of large, web-mined text datasets covering hundreds of languages. However, to date there has been no systematic analysis of the quality of these publicly available datasets, or whether the datasets actually contain content in the languages they claim to represent. In this work, we manually audit the quality of 205 language-specific corpora released with five major public datasets (CCAligned, ParaCrawl, WikiMatrix, OSCAR, mC4), and audit the correctness of language codes in a sixth (JW300). We find that lower-resource corpora have systematic issues: at least 15 corpora are completely erroneous, and a significant fraction contains less than 50% sentences of acceptable quality. Similarly, we find 82 corpora that are mislabeled or use nonstandard/ambiguous language codes. We demonstrate that these issues are easy to detect even for non-speakers of the languages in question, and supplement the human judgements with automatic analyses. Inspired by our analysis, we recommend techniques to evaluate and improve multilingual corpora and discuss the risks that come with low-quality data releases.