In this paper, we propose a novel pretraining-based encoder-decoder framework, which can generate the output sequence based on the input sequence in a two-stage manner. For the encoder of our model, we encode the input sequence into context representations using BERT. For the decoder, there are two stages in our model, in the first stage, we use a Transformer-based decoder to generate a draft output sequence. In the second stage, we mask each word of the draft sequence and feed it to BERT, then by combining the input sequence and the draft representation generated by BERT, we use a Transformer-based decoder to predict the refined word for each masked position. To the best of our knowledge, our approach is the first method which applies the BERT into text generation tasks. As the first step in this direction, we evaluate our proposed method on the text summarization task. Experimental results show that our model achieves new state-of-the-art on both CNN/Daily Mail and New York Times datasets.

Brown-Forman, whose brands include Old Forester and Woodford Reserve bourbon, has spent the past three years taking inventory and integrating diverse pools of consumer, production and sales data across its global operations, as part of a broader effort to update an aging technology stack, Mr. Nall said. That was no small task. Founded nearly 150 year ago, Brown-Forman today has some 4,800 employees and operates in more than 170 countries world-wide. Since becoming CIO in 2015, Mr. Nall has led a gradual strategic shift in the role of the company's enterprise information-technology hub, from a backroom tech support service to a business partner aligned with marketing and sales teams, as well as other corporate and global production functions. That shift has seen data scientists and other IT pros increasingly working across the entire business on efforts to drive efficiencies and generate revenue: "Technology is interwoven into the whole process," he said. Nowhere is the need for a more business-oriented IT model more clear than with the emerging powers of artificial intelligence and machine learning to supercharge corporate decision-making, he said.

Naval technology is developing so rapidly that Australia's new $50 billion fleet of submarines may one day have to face deadly underwater drones, an expert has warned. Earlier this month, the federal government announced the signing of the Attack class submarine Strategic Partnering Agreement with French shipbuilder Naval Group. It will build 12 attack submarines to replace the Royal Australian Navy's ageing Collins class vessels, with the first one scheduled to be delivered in the early 2030s, the federal government said. But Russia has already provided a glimpse of underwater autonomous – or drone - weaponry. The Russian Ministry of Defence released testing footage of its'Poseidon' – a high-speed nuclear torpedo. Naval chiefs said the weapon is capable of carrying both conventional and nuclear warheads and will have a maximum speed of 200 km/h.

Which AI applications are playing a role in automation or augmentation of the retail process? How are retail companies using these technologies to stay ahead of their competitors today, and what innovations are being pioneered as potential retail game-changers over the next decade? Innovation is a double-edged sword, and as with any innovation results are a mixed bag. While many AI applications have yielded increased ROI--this case study of AI in retail marketing segmentation is one example--others have been tried and failed to meet expectations, shining a light on barriers that still need to be overcome before such innovations become industry drivers. Below are 10 brief use cases across five retail domains or phases.

"We are not talking about Terminator. We're talking about much simpler technologies, which are at best a few years away, and in fact, many of which you can see under development today in every theater of the war." He spoke February 14th as part of discussion called Killer Robots: Technological, Legal and Ethical Challenges at a meeting of the American Association for the Advancement of Science. "And so these are systems that are using sensors and software processing on their own to determine what constitutes a target and then applying lethal force to that, without supervision or meaningful human control." Another speaker, Peter Asaro, co-founder of the International Committee for Robot Arms Control, has participated in U.N. talks on autonomous weapons.

If you shop at Westfield, you've probably been scanned and recorded by dozens of hidden cameras built into the centres' digital advertising billboards. The semi-camouflaged cameras can determine not only your age and gender but your mood, cueing up tailored advertisements within seconds, thanks to facial detection technology. Westfield's Smartscreen network was developed by the French software firm Quividi back in 2015. Their discreet cameras capture blurry images of shoppers and apply statistical analysis to identify audience demographics. And once the billboards have your attention they hit record, sharing your reaction with advertisers.

Many applications in Bayesian statistics are extremely computationally intensive. However, they are also often inherently parallel, making them prime targets for modern massively parallel central processing unit (CPU) architectures. While the use of multi-core and distributed computing is widely applied in the Bayesian community, very little attention has been given to fine-grain parallelisation using single instruction multiple data (SIMD) operations that are available on most modern commodity CPUs. Rather, most fine-grain tuning in the literature has centred around general purpose graphics processing units (GPGPUs). Since the effective utilisation of GPGPUs typically requires specialised programming languages, such technologies are not ideal for the wider Bayesian community. In this work, we practically demonstrate, using standard programming libraries, the utility of the SIMD approach for several topical Bayesian applications. In particular, we consider sampling of the prior predictive distribution for approximate Bayesian computation (ABC), and the computation of Bayesian $p$-values for testing prior weak informativeness. Through minor code alterations, we show that SIMD operations can improve the floating point arithmetic performance resulting in up to $6\times$ improvement in the overall serial algorithm performance. Furthermore $4$-way parallel versions can lead to almost $19\times$ improvement over a na\"{i}ve serial implementation. We illustrate the potential of SIMD operations for accelerating Bayesian computations and provide the reader with essential implementation techniques required to exploit modern massively parallel processing environments using standard software development tools.

Adaptive gradient methods such as Adam have been shown to be very effective for training deep neural networks (DNNs) by tracking the second moment of gradients to compute the individual learning rates. Differently from existing methods, we make use of the most recent first moment of gradients to compute the individual learning rates per iteration. The motivation behind it is that the dynamic variation of the first moment of gradients may provide useful information to obtain the learning rates. We refer to the new method as the rapidly adapting moment estimation (RAME). The theoretical convergence of deterministic RAME is studied by using an analysis similar to the one used in [1] for Adam. Experimental results for training a number of DNNs show promising performance of RAME w.r.t. the convergence speed and generalization performance compared to the stochastic heavy-ball (SHB) method, Adam, and RMSprop.

Like early mammals scuttering between the legs of tyrannosaurs, a lot of little companies are trying to weave around--and maybe even outlast--the big boys of self-driving technology. One such example is Perrone Robotics, a small Virginia company that has developed a self-driving package that it says can be quickly adapted to any vehicle. This Swiss Army knife of an AI can give smarts to an existing car, shuttle bus, or truck--even the gargantuan trucks used in mining. Tiny shuttles and behemoth trucks sell in small numbers, and equipping them to drive themselves is beneath the dignity of major players, like Alphabet's Waymo and General Motors' Cruise Automation. "What we're doing, certainly Waymo and GM Cruise could do, but they are focused on their own agenda. This is our niche, and we are going where we can add real value," says David Hofert, the chief marketing officer at Perrone Robotics.

Think of a university campus: it has its own roads, shops, residential areas, banks and transport links. It may be visited by tens of thousands of people each day. It is, in effect, a tiny city. Across the globe, these mini metropolises are increasingly opting for a smart city approach. This is a tech-driven model that's used in places such as Barcelona, where street lamps react intelligently to surroundings to save energy; Seattle, where smart traffic lights respond to the conditions on the road; and even Milton Keynes, which has a real-time "data hub" sharing information about the town's energy and water consumption, transport, weather and pollution.