In recent years there has been a sharp rise in networking applications, in which significant events need to be classified but only a few training instances are available. These are known as cases of one-shot learning. Examples include analyzing network traffic under zero-day attacks, and computer vision tasks by sensor networks deployed in the field. To handle this challenging task, organizations often use human analysts to classify events under high uncertainty. Existing algorithms use a threshold-based mechanism to decide whether to classify an object automatically or send it to an analyst for deeper inspection. However, this approach leads to a significant waste of resources since it does not take the practical temporal constraints of system resources into account. Our contribution is threefold. First, we develop a novel Deep Reinforcement One-shot Learning (DeROL) framework to address this challenge. The basic idea of the DeROL algorithm is to train a deep-Q network to obtain a policy which is oblivious to the unseen classes in the testing data. Then, in real-time, DeROL maps the current state of the one-shot learning process to operational actions based on the trained deep-Q network, to maximize the objective function. Second, we develop the first open-source software for practical artificially intelligent one-shot classification systems with limited resources for the benefit of researchers in related fields. Third, we present an extensive experimental study using the OMNIGLOT dataset for computer vision tasks and the UNSW-NB15 dataset for intrusion detection tasks that demonstrates the versatility and efficiency of the DeROL framework.

Although artificial intelligence has been around since the 1950s yet it is definitely not late to get started…no matter if you are a developer or an enterprise manager. Actually, given the (major) developments in the past few years, you couldn't be in a better time to get started in Artificial Intelligence (AI). AI is redefining the experiences humans have with machines and enhancing even richer experiences for end users and entities alike. This entry is a follow-up to the talk I delivered during the Google Cloud Day in Malta on the 26th July 2018. Below, I will first set the context and then outline a selection of latest developments that should motivate you to get started in artificial intelligence.

As we saw in the article about improvements in Deep Q Learning, value-based methods have high variability. To reduce this problem, we spoke about using the advantage function instead of the value function. This function will tell us the improvement compared to the average the action taken at that state is. In other words, this function calculates the extra reward I get if I take this action. The extra reward is that beyond the expected value of that state.

Algorithms created at Stevens Institute of Technology in New Jersey can teach robots to adapt to changing conditions related to protecting and preserving underwater infrastructure. Researchers at Stevens Institute of Technology in New Jersey have created algorithms to teach robots to adapt to changing conditions related to protecting and preserving underwater infrastructure. Stevens' Brendan Englot leads a group that uses reinforcement learning algorithms trained on sonar data. The group's robots emit high-frequency chirps and measure how long it takes the sound to return after reflecting off surrounding structures, gathering data and acquiring situational awareness while various forces buffet them. The research team recently dispatched a robot to autonomously map a Manhattan pier without a prior model.

OpenAI has trained a human-like robot hand to manipulate physical objects with unprecedented dexterity. Their system, called Dactyl, is trained entirely in simulation and transfers its knowledge to reality, adapting to real-world physics. Dactyl learns from scratch using the same general-purpose reinforcement learning algorithm and code as OpenAI Five. The results show that it's possible to train agents in simulation and have them solve real-world tasks, without physically-accurate modeling of the world. Dactyl is a system for manipulating objects using a Shadow Dexterous Hand.

An artificial-intelligence approach created at the University of North Carolina at Chapel Hill Eshelman School of Pharmacy can teach itself to design new drug molecules from scratch and has the potential to dramatically accelerate the design of new drug candidates. The system is called Reinforcement Learning for Structural Evolution, known as ReLeaSE, and is an algorithm and computer program that comprises two neural networks which can be thought of as a teacher and a student. The teacher knows the syntax and linguistic rules behind the vocabulary of chemical structures for about 1.7 million known biologically active molecules. By working with the teacher, the student learns over time and becomes better at proposing molecules that are likely to be useful as new medicines. The University has applied for a patent for the technology, and the team published a proof-of-concept study in the journal Science Advances last week.

This work demonstrates the potential of deep reinforcement learning techniques for transmit power control in emerging and future wireless networks. Various techniques have been proposed in the literature to find near-optimal power allocations, often by solving a challenging optimization problem. Most of these algorithms are not scalable to large networks in real-world scenarios because of their computational complexity and instantaneous cross-cell channel state information (CSI) requirement. In this paper, a model-free distributed dynamic power allocation scheme is developed based on deep reinforcement learning. Each transmitter collects CSI and quality of service (QoS) information from several neighbors and adapts its own transmit power accordingly. The objective is to maximize a weighted sum-rate utility function, which can be particularized to achieve maximum sum-rate or proportionally fair scheduling (with weights that are changing over time). Both random variations and delays in the CSI are inherently addressed using deep Q-learning. For a typical network architecture, the proposed algorithm is shown to achieve near-optimal power allocation in real time based on delayed CSI measurements available to the agents. This work indicates that deep reinforcement learning based radio resource management can be very fast and deliver highly competitive performance, especially in practical scenarios where the system model is inaccurate and CSI delay is non-negligible.

We formulate simple assumptions, implying the Robbins-Monro conditions for the $Q$-learning algorithm with the local learning rate, depending on the number of visits of a particular state-action pair (local clock) and the number of iteration (global clock). It is assumed that the Markov decision process is communicating and the learning policy ensures the persistent exploration. The restrictions are imposed on the functional dependence of the learning rate on the local and global clocks. The result partially confirms the conjecture of Bradkte (1994).

Noname manuscript No. (will be inserted by the editor) Abstract Artificial intelligence recently had a great advancements caused by the emergence of new processing power and machine learning methods. Having said that, the learning capability of artificial intelligence is still at its infancy comparing to the learning capability of human and many animals. Many of the current artificial intelligence applications can only operate in a very orchestrated, specific environments with an extensive training set that exactly describes the conditions that will occur during execution time. Having that in mind, and considering the several existing machine learning methods this question rises that'What are some of the best ways for a machine to learn?' Regarding the learning methods of human, Confucius' point of view is that they are by experience, imitation and reflection. This paper tries to explore and discuss regarding these three ways of learning and their implementations in machines by having a look at how they happen in minds. Keywords Artificial Intelligence · Supervised Learning · Reinforcement Learning · Unsupervised Learning · Machine Imagination · Machine Learning · Cognitive Development 1 Introduction How minds work, or in another word how a human brain thinks, with the goal of implementing it in machines, is a long-term question in artificial intelligence.

We use reinforcement learning (RL) to learn dexterous in-hand manipulation policies which can perform vision-based object reorientation on a physical Shadow Dexterous Hand. The training is performed in a simulated environment in which we randomize many of the physical properties of the system like friction coefficients and an object's appearance. Our policies transfer to the physical robot despite being trained entirely in simulation. Our method does not rely on any human demonstrations, but many behaviors found in human manipulation emerge naturally, including finger gaiting, multi-finger coordination, and the controlled use of gravity. Our results were obtained using the same distributed RL system that was used to train OpenAI Five. We also include a video of our results: https://youtu.be/jwSbzNHGflM