When should an online reinforcement learning-based frequency agile cognitive radar be expected to outperform a rule-based adaptive waveform selection strategy? We seek insight regarding this question by examining a dynamic spectrum access scenario, in which the radar wishes to transmit in the widest unoccupied bandwidth during each pulse repetition interval. Online learning is compared to a fixed rule-based sense-and-avoid strategy. We show that given a simple Markov channel model, the problem can be examined analytically for simple cases via stochastic dominance. Additionally, we show that for more realistic channel assumptions, learning-based approaches demonstrate greater ability to generalize. However, for short time-horizon problems that are well-specified, we find that machine learning approaches may perform poorly due to the inherent limitation of convergence time. We draw conclusions as to when learning-based approaches are expected to be beneficial and provide guidelines for future study.

Who can take this course? Unfortunately, learners residing in one or more of the following countries or regions will not be able to register for this course: Iran, Cuba and the Crimea region of Ukraine. While edX has sought licenses from the U.S. Office of Foreign Assets Control (OFAC) to offer our courses to learners in these countries and regions, the licenses we have received are not broad enough to allow us to offer this course in all locations. Who can take this course?

Check out the on-demand sessions from the Low-Code/No-Code Summit to learn how to successfully innovate and achieve efficiency by upskilling and scaling citizen developers. Last weekend may have been a holiday in Silicon Valley, with AI researchers wolfing down turkey before preparing to fly to New Orleans for the start of NeurIPS (which one researcher called an "annual gala" of AI and another "Burning Man" for AI). But nothing seems to stop the pace of news -- or the debate -- about AI models and research, even Thanksgiving in the U.S. My question: Is it all moving too fast and furiously for responsible and ethical AI efforts to keep up? For example, it was the end of the day on November 23rd -- a time when most Americans were likely in holiday travel mode -- when Stability AI announced the release of Stable Diffusion 2.0. The announcement was an updated version of its open-source text-to-image generator, which immediately became wildly popular when it was released just three months ago.

Minesweeper is an interesting single player game based on logic, memory and guessing. Solving Minesweeper has been shown to be an NP-hard task. Deterministic solvers are the best known approach for solving Minesweeper. This project proposes a neural network based learner for solving Minesweeper. To choose the best learner, different architectures and configurations of neural networks were trained on hundreds of thousands of games. Surprisingly, the proposed neural network based learner has shown to be a very good approximation function for solving Minesweeper. The neural network learner competes well with the CSP solvers, especially in Beginner and Intermediate modes of the game. It was also observed that despite having high success rates, the best neural learner was considerably slower than the best deterministic solver. This report also discusses the overheads and limitations faced while creating highly successful neural networks for Minesweeper.

The word alignment task, despite its prominence in the era of statistical machine translation (SMT), is niche and under-explored today. In this two-part tutorial, we argue for the continued relevance for word alignment. The first part provides a historical background to word alignment as a core component of the traditional SMT pipeline. We zero-in on GIZA++, an unsupervised, statistical word aligner with surprising longevity. Jumping forward to the era of neural machine translation (NMT), we show how insights from word alignment inspired the attention mechanism fundamental to present-day NMT. The second part shifts to a survey approach. We cover neural word aligners, showing the slow but steady progress towards surpassing GIZA++ performance. Finally, we cover the present-day applications of word alignment, from cross-lingual annotation projection, to improving translation.

Several policy options exist, or have been proposed, to further responsible artificial intelligence (AI) development and deployment. Institutions, including U.S. government agencies, states, professional societies, and private and public sector businesses, are well positioned to implement these policies. However, given limited resources, not all policies can or should be equally prioritized. We define and review nine suggested policies for furthering responsible AI, rank each policy on potential use and impact, and recommend prioritization relative to each institution type. We find that pre-deployment audits and assessments and post-deployment accountability are likely to have the highest impact but also the highest barriers to adoption. We recommend that U.S. government agencies and companies highly prioritize development of pre-deployment audits and assessments, while the U.S. national legislature should highly prioritize post-deployment accountability. We suggest that U.S. government agencies and professional societies should highly prioritize policies that support responsible AI research and that states should highly prioritize support of responsible AI education. We propose that companies can highly prioritize involving community stakeholders in development efforts and supporting diversity in AI development. We advise lower levels of prioritization across institutions for AI ethics statements and databases of AI technologies or incidents. We recognize that no one policy will lead to responsible AI and instead advocate for strategic policy implementation across institutions.

We have developed a model for online continual or lifelong reinforcement learning (RL) inspired on the insect brain. Our model leverages the offline training of a feature extraction and a common general policy layer to enable the convergence of RL algorithms in online settings. Sharing a common policy layer across tasks leads to positive backward transfer, where the agent continuously improved in older tasks sharing the same underlying general policy. Biologically inspired restrictions to the agent's network are key for the convergence of RL algorithms. This provides a pathway towards efficient online RL in resource-constrained scenarios.

Surrogate models have shown to be an extremely efficient aid in solving engineering problems that require repeated evaluations of an expensive computational model. They are built by sparsely evaluating the costly original model and have provided a way to solve otherwise intractable problems. A crucial aspect in surrogate modelling is the assumption of smoothness and regularity of the model to approximate. This assumption is however not always met in reality. For instance in civil or mechanical engineering, some models may present discontinuities or non-smoothness, e.g., in case of instability patterns such as buckling or snap-through. Building a single surrogate model capable of accounting for these fundamentally different behaviors or discontinuities is not an easy task. In this paper, we propose a three-stage approach for the approximation of non-smooth functions which combines clustering, classification and regression. The idea is to split the space following the localized behaviors or regimes of the system and build local surrogates that are eventually assembled. A sequence of well-known machine learning techniques are used: Dirichlet process mixtures models (DPMM), support vector machines and Gaussian process modelling. The approach is tested and validated on two analytical functions and a finite element model of a tensile membrane structure.

In this article, we plan to provide an introduction about some basics about robots for readers. Several key topics of classic robotics will be introduced, including robot representation, robot rotational motion, coordinates transformation and velocity transformation. By now, classic rigid-body robot analysis is still the main-stream approach in robot controlling and motion planning. In this article, no data-driven or machine learning based methods will be introduced. Most of the materials covered in this article are based on the rigid-body kinematics that the readers probably have learned from the physics course at high-school or college. Meanwhile, these classic robot kinematics analyses will serve as the foundation for the latest intelligent robot control algorithms in modern robotics studies.

Abstract--Partial label learning (PLL) is an important problem that allows each training example to be labeled with a coarse candidate set, which well suits many real-world data annotation scenarios with label ambiguity. Despite the promise, the performance of PLL often lags behind the supervised counterpart. Specifically, our proposed framework PiCO consists of a contrastive learning module along with a novel class prototype-based label disambiguation algorithm. PiCO produces closely aligned representations for examples from the same classes and facilitates label disambiguation. Theoretically, we show that these two components are mutually beneficial, and can be rigorously justified from an expectation-maximization (EM) algorithm perspective. Moreover, we study a challenging yet practical noisy partial label learning setup, where the ground-truth may not be included in the candidate set. To remedy this problem, we present an extension PiCO+ that performs distance-based clean sample selection and learns robust classifiers by a semi-supervised contrastive learning algorithm. Extensive experiments demonstrate that our proposed methods significantly outperform the current state-of-the-art approaches in standard and noisy PLL tasks and even achieve comparable results to fully supervised learning.