Deep reinforcement learning has been successfully used in many dynamic decision making domains, especially those with very large state spaces. However, it is also well-known that deep reinforcement learning can be very slow and resource intensive. The resulting system is often brittle and difficult to explain. In this paper, we attempt to address some of these problems by proposing a framework of Rule-interposing Learning (RIL) that embeds high level rules into the deep reinforcement learning. With some good rules, this framework not only can accelerate the learning process, but also keep it away from catastrophic explorations, thus making the system relatively stable even during the very early stage of training. Moreover, given the rules are high level and easy to interpret, they can be easily maintained, updated and shared with other similar tasks.

In this framework, the robot learning problem corresponds to an RL problem that aims at obtaining a policy π: S G A such that the expected discounted sum of rewards is maximized for any given goal. When the reward function is sparse, as assumed here, this RL problem is particularly hard to solve. In particular, we consider here reward functions that are described as follows: R ( s,a,s null,g) 1[ d( s null,g) null R] 1 where 1 is the indicator function, d is a distance, and null R 0 is a fixed threshold. To tackle this issue, Andrychowicz et al. [2017] proposed HER, which is based on the following principle: Any trajectory that failed to reach its goal still carries useful information; it has at least reached the states of its trajectory path. Using this natural and powerful idea, memory replay can be augmented with the failed trajectories by changing their goals in hindsight .

The potential for machine learning to disrupt the medical profession is the subject of ongoing debate within biomedical informatics. This study aimed to explore psychiatrists' opinions about the potential impact of innovations in artificial intelligence and machine learning on psychiatric practice. In Spring 2019, we conducted a web-based survey of 791 psychiatrists from 22 countries worldwide. The survey measured opinions about the likelihood future technology would fully replace physicians in performing ten key psychiatric tasks. This study involved qualitative descriptive analysis of written response to three open-ended questions in the survey. Comments were classified into four major categories in relation to the impact of future technology on patient-psychiatric interactions, the quality of patient medical care, the profession of psychiatry, and health systems. Overwhelmingly, psychiatrists were skeptical that technology could fully replace human empathy. Many predicted that 'man and machine' would increasingly collaborate in undertaking clinical decisions, with mixed opinions about the benefits and harms of such an arrangement. Participants were optimistic that technology might improve efficiencies and access to care, and reduce costs. Ethical and regulatory considerations received limited attention. This study presents timely information of psychiatrists' view about the scope of artificial intelligence and machine learning on psychiatric practice. Psychiatrists expressed divergent views about the value and impact of future technology with worrying omissions about practice guidelines, and ethical and regulatory issues.

Beating humans in a penny-matching game by leveraging cognitive hierarchy theory and Bayesian learning Ran Tian, Nan Li, Ilya Kolmanovsky, and Anouck Girard Abstract -- It is a longstanding goal of artificial intelligence (AI) to be superior to human beings in decision making. Games are suitable for testing AI capabilities of making good decisions in non-numerical tasks. In this paper, we develop a new AI algorithm to play the penny-matching game considered in Shannon's "mind-reading machine" (1953) against human players. In particular, we exploit cognitive hierarchy theory and Bayesian learning techniques to continually evolve a model for predicting human player decisions, and let the AI player make decisions according to the model predictions to pursue the best chance of winning. Experimental results show that our AI algorithm beats 27 out of 30 volunteer human players. I NTRODUCTION Developing artificial intelligence (AI) to beat humans in strategic games has been drawing attention/interest of researchers for decades [1]-[10].

A BSTRACT Increasing model size when pretraining natural language representations often results in improved performance on downstream tasks. However, at some point further model increases become harder due to GPU/TPU memory limitations, longer training times, and unexpected model degradation. To address these problems, we present two parameter-reduction techniques to lower memory consumption and increase the training speed of BERT (Devlin et al., 2019). Comprehensive empirical evidence shows that our proposed methods lead to models that scale much better compared to the original BERT. We also use a self-supervised loss that focuses on modeling inter-sentence coherence, and show it consistently helps downstream tasks with multi-sentence inputs. As a result, our best model establishes new state-of-the-art results on the GLUE, RACE, and SQuAD benchmarks while having fewer parameters compared to BERT -large. The code and the pretrained models are available at https://github.com/ Many nontrivial NLP tasks, including those that have limited training data, have greatly benefited from these pre-trained models. One of the most compelling signs of these breakthroughs is the evolution of machine performance on a reading comprehension task designed for middle and highschool English exams in China, the RACE test (Lai et al., 2017): the paper that originally describes the task and formulates the modeling challenge reports then state-of-the-art machine accuracy at 44. 1%; the latest published result reports their model performance at 83. 2% (Liu et al., 2019); the work we present here pushes it even higher to 89 .4%, a stunning 45 .3% Evidence from these improvements reveals that a large network is of crucial importance for achieving state-of-the-art performance (Devlin et al., 2019; Radford et al., 2019). It has become common practice to pre-train large models and distill them down to smaller ones (Sun et al., 2019; Turc et al., 2019) for real applications.

Systems engineering processes coordinate the effort of different individuals to generate a product satisfying certain requirements. As the involved engineers are self-interested agents, the goals at different levels of the systems engineering hierarchy may deviate from the system-level goals which may cause budget and schedule overruns. Therefore, there is a need of a systems engineering theory that accounts for the human behavior in systems design. To this end, the objective of this paper is to develop and analyze a principal-agent model of a one-shot (single iteration), shallow (one level of hierarchy) systems engineering process. We assume that the systems engineer maximizes the expected utility of the system, while the subsystem engineers seek to maximize their expected utilities. Furthermore, the systems engineer is unable to monitor the effort of the subsystem engineer and may not have a complete information about their types or the complexity of the design task. However, the systems engineer can incentivize the subsystem engineers by proposing specific contracts. To obtain an optimal incentive, we pose and solve numerically a bi-level optimization problem. Through extensive simulations, we study the optimal incentives arising from different system-level value functions under various combinations of effort costs, problem-solving skills, and task complexities.

Though video game culture is seldom a quiet, peaceful place, the uproar over Blizzard Entertainment punishing a popular gamer for showing support for Hong Kong protesters has shaken the whole industry. Ng Wai Chung, a Hearthstone player from Hong Kong who goes by the name "Blitzchung," championed the pro-democracy protests in his hometown that have raged for the past five months during his appearance on a post-game stream. And Blizzard, the developer and publisher of Hearthstone, quickly responded with blanket punishments for everyone involved. It's the latest example of an American company caught between business interests in China and western-world freedom of speech. Outrage over Blizzard's reaction swiftly came from players, industry titans and politicians.

On a cold, sunny October day on the outskirts of Copenhagen, Denmark, a group of men dressed in black gathers outside Brondby Stadium to shoot off a couple of rockets, raise their fists and shout about how the home team will soon beat -- and beat up -- the visiting archnemesis, FC Copenhagen. Police are out in force, riot helmets at the ready. Brondby-Copenhagen matches have a history of leading to vandalism, arrests and general mayhem. An attempted photo of the group gets a gloved hand in the face. "You need to stop," says the hand's black-clad owner, before he disappears back into the crowd.

The rapid rate of change enabled by software make this industry more vulnerable than most to the falling behind on the innovation curve. This problem has only accelerated in recent years as the number of disruptive technologies have grown at an exponential rate fueled by the growing size of the market and the number of software engineers. The open source community has been a driving source of disruptive technologies such as big data Hadoop and Spark, JavaScript frameworks like Angular and React, and machine learning frameworks like TensorFlow. Software based companies who do not embrace these disruptive technologies face the ever-increasing risk of being pushed aside by those that do. To make this even more challenging, the skills required to enhance the current product and the skills required to innovate using new disruptive technologies are different.

In June, a video allegedly showing Datuk Seri Azmin Ali, the Malaysian minister of economic affairs, engaged in a sexual tryst with Muhammad Haziq Abdul Aziz, a deputy Malaysian minister's secretary, surfaced online. The video spread like wildfire, and subsequently threw the country's media into a frenzy. The video had real-world consequences, and Abdul Aziz, who in the eyes of the government had committed a crime, was quickly arrested. But, according to Malaysia's prime minister, the video was just one of countless other scarily-accurate deepfake videos that have been finding their way onto the internet in the last year. Deepfakes work by using something called a generative adversarial network (GAN), which is made up of two artificial intelligent processes that are pitted against each other – a generator and a discriminator.