Adaptive optimization algorithms such as Adam (Kingma & Ba, 2014) are widely used in deep learning. The stability of such algorithms is often improved with a warmup schedule for the learning rate. Motivated by the difficulty of choosing and tuning warmup schedules, Liu et al. (2019) propose automatic variance rectification of Adam's adaptive learning rate, claiming that this rectified approach ("RAdam") surpasses the vanilla Adam algorithm and reduces the need for expensive tuning of Adam with warmup. In this work, we point out various shortcomings of this analysis. We then provide an alternative explanation for the necessity of warmup based on the magnitude of the update term, which is of greater relevance to training stability. Finally, we provide some "rule-of-thumb" warmup schedules, and we demonstrate that simple untuned warmup of Adam performs more-or-less identically to RAdam in typical practical settings. We conclude by suggesting that practitioners stick to linear warmup with Adam, with a sensible default being linear warmup over $2 / (1 - \beta_2)$ training iterations.

Recent policy optimization approaches have achieved substantial empirical success by constructing surrogate optimization objectives. The Approximate Policy Iteration objective (Schulman et al., 2015a; Kakade and Langford, 2002) has become a standard optimization target for reinforcement learning problems. Using this objective in practice requires an estimator of the advantage function. Policy optimization methods such as those proposed in Schulman et al. (2015b) estimate the advantages using a parametric critic. In this work we establish conditions under which the parametric approximation of the critic does not introduce bias to the updates of surrogate objective. These results hold for a general class of parametric policies, including deep neural networks. We obtain a result analogous to the compatible features derived for the original Policy Gradient Theorem (Sutton et al., 1999). As a result, we also identify a previously unknown bias that current state-of-the-art policy optimization algorithms (Schulman et al., 2015a, 2017) have introduced by not employing these compatible features.

Recent policy optimization approaches (Schulman et al., 2015a, 2017) have achieved substantial empirical successes by constructing new proxy optimization objectives. These proxy objectives allow stable and low variance policy learning, but require small policy updates to ensure that the proxy objective remains an accurate approximation of the target policy value. In this paper we derive an alternative objective that obtains the value of the target policy by applying importance sampling. This objective can be directly estimated from samples, as it takes an expectation over trajectories generated by the current policy. However, the basic importance sampled objective is not suitable for policy optimization, as it incurs unacceptable variance. We therefore introduce an approximation that allows us to directly trade-off the bias of approximation with the variance in policy updates. We show that our approximation unifies the proxy optimization approaches with the importance sampling objective and allows us to interpolate between them. We then provide a theoretical analysis of the method that directly quantifies the error term due to the approximation. Finally, we obtain a practical algorithm by optimizing the introduced objective with proximal policy optimization techniques (Schulman etal., 2017). We empirically demonstrate that the result-ing algorithm yields superior performance on continuous control benchmarks

In the 1920s, the Soviet scientist Ilya Ivanovich Ivanov used artificial insemination to breed a'humanzee' – a cross between a human and our closest relative species, the chimpanzee. Given the moral quandaries a humanzee might create, we can be thankful that Ivanov failed: when the winds of Soviet scientific preferences changed, he was arrested and exiled. But Ivanov's endeavour points to the persistent, post-Darwinian fear and fascination with the question of whether humans are a creature apart, above all other life, or whether we're just one more animal in a mad scientist's menagerie. Humans have searched and repeatedly failed to rescue ourselves from this disquieting commonality. Numerous dividers between humans and beasts have been proposed: thought and language, tools and rules, culture, imitation, empathy, morality, hate, even a grasp of'folk' physics. But they've all failed, in one way or another. I'd like to put forward a new contender – strangely, the very same tendency that elicits the most dread and excitement among political and economic commentators today.

When being empathetic is your full-time job, burning out is only human. Few people are more aware of this than customer service representatives, who are tasked with approaching each conversation with energy and compassion -- whether it's their first call of the day or their 60th. It's their job to make even the most difficult customer feel understood and respected while still providing them accurate information. But over the last few years, an unlikely aide has come forward: artificial intelligence tools designed to help people tap into and maintain "human" characteristics like empathy and compassion. One of these tools is a platform called Cogito, named for the famous Descartes philosophy Cogito, ergo sum ("I think, therefore I am").

The ads you see online could soon get harder to refuse, thanks to a new artificial intelligence (AI) system that predicts whether you'll like an ad before it has even run. Designed by a technical team in Brisbane and delivered to marketers through headquarters in Austin, Texas, Junction AI technology has hit a nerve in an industry where new display ads often fall flat after advertisers pay handsomely to place them in key outlets. Copywriters lean heavily on ad-writing techniques like A/B testing, but these only go so far in predicting whether online citizens will respond to a particular ad. This leaves advertisers all but guessing whether they have chosen the right words and images to convince potential new customers to click through. That's a challenge for marketers that are increasingly equipping content marketing organisations (CMOs) to drive deeper engagement with customers and prospects in an ever more-crowded advertising market expected to surge from $US226.6b

GNW The global artificial intelligence (AI) in retail market attained $720.0 million in 2018 and is predicted to witness a CAGR of 35.4% during 2019–2024 (forecast period). The factors contributing to the growth of the market include the increasing investments in AI by retail companies and expanding e-retail industry. With AI, retailers have been able to automate their work processes, study consumer behavior, and capture relevant data through the adoption of numerous advanced technologies, such as machine learning, natural language processing (NLP), and computer vision. When technology is considered, the AI in retail market is divided into computer vision, NLP, machine learning, and others (which include gesture recognition and analytics). Machine learning generated the highest revenue during the historical period (2014–2018) and is expected to dominate the market during the forecast period as well.

Australian singer Cody Simpson is currently making headlines globally for going public with his relationship with Miley Cyrus. And as the star becomes the talk of the town, Masked Singer Australia fans are now convinced he is the mystery Robot contestant. On Monday night's episode, a fresh batch of clues added to the growing pile of evidence that the iYiYi hit-maker is behind the elaborate costume. Cody Simpson (left) is The Masked Singer's Robot (right) according to a growing pile of evidence Cody has sung three times on the show so far, and remains in the competition against six celebrity rivals. With the clues mounting, fans believe the mystery singer is definitely Cody.

Scott Sanchez used to have a hard time deciding what to eat, especially when he was traveling. The 42-year-old wanted to lose weight and found he needed to dissect a menu with the waiter before he could order. It was a challenge, he says, but one that gave birth to The Fit, a menu personalization platform that uses artificial intelligence to give restaurant brands and their customers the option to customize their menu and food choices. At least 32 million Americans -- including 5 million children -- have food allergies, according to nonprofit Food Allergy Research & Education. Whenever they eat out, they need to make sure there are no ingredients in the food that could trigger an allergic reaction. Those with dietary preferences, from vegans to people wanting to lose weight -- there are 93 million obese people in the U.S., according to the Centers for Disease Control and Prevention -- also need to carefully examine menus.

Ever since Homo erectus, or upright man, a type of early human, carved a piece of stone into a tool, the welfare of our species has been on the increase. Indeed, this technological breakthrough led first to the hand ax, and eventually to the iPhone. We have found it convenient to organize the most dramatic periods of change between these inventions into four industrial revolutions. As each revolution unfolded, dire predictions of massive job losses ensued. Looking back at the first three, we can see how the concerns were misplaced. The number of jobs increased each time, as did living standards.