Reinforcement learning (RL) is widely used in applications where one needs to perform sequential decision-making while interacting with the environment. The standard RL problem with safety constraints is generally mathematically modeled by constrained Markov Decision Processes (CMDP), which is linear in objective and rules in occupancy measure space, where the problem becomes challenging in the case where the model is unknown apriori. The problem further becomes challenging when the decision requirement includes optimizing a concave utility while satisfying some nonlinear safety constraints. To solve such a nonlinear problem, we propose a conservative stochastic primal-dual algorithm (CSPDA) via a randomized primal-dual approach. By leveraging a generative model, we prove that CSPDA not only exhibits Õ(1/ε2)sample complexity, but also achieves zero constraint violations for the concave utility CMDP. Compared with the previous works, the best available sample complexity for CMDP with zero constraint violation is Õ(1/ε5). Hence, the proposed algorithm provides a significant improvement as compared to the state-of-the-art.

We study the estimation of risk-sensitive policies in reinforcement learning problems defined by a Markov Decision Process (MDPs) whose state and action spaces are countably finite. Prior efforts are predominately afflicted by computational challenges associated with the fact that risk-sensitive MDPs are time-inconsistent. To ameliorate this issue, we propose a new definition of risk, which we call caution, as a penalty function added to the dual objective of the linear programming (LP) formulation of reinforcement learning. The caution measures the distributional risk of a policy, which is a function of the policy's long-term state occupancy distribution. To solve this problem in an online model-free manner, we propose a stochastic variant of primal-dual method that uses Kullback-Lieber (KL) divergence as its proximal term. We establish that the number of iterations/samples required to attain approximately optimal solutions of this scheme matches tight dependencies on the cardinality of the state and action spaces, but differs in its dependence on the infinity norm of the gradient of the risk measure. Experiments demonstrate the merits of this approach for improving the reliability of reward accumulation without additional computational burdens.

Machine learning and artificial intelligence are near the top of the list of items dominating discussions about digital transformation. Chris Bedi is the CIO at ServiceNow and he said, during a briefing at the company's Knowledge 18 event, CEOs are now value in speed over cost. As businesses are changing, he says there's a huge sense of urgency as companies want to ensure they're not left behind. When it comes to AI and machine learning, Bedi said "What a lot of companies struggle with is giving it meaning. It really boils down to three things: Speed - am I really helping my company fundamentally operate faster; Ingtelligence - am I getting smarter as a company? We should never have dashboard that doesn't after a suggestion to the human and; Experience".

Amazon Web Services (AWS), the cloud computing arm of ecommerce giant Amazon, plans to make inroads into the over-the-top (OTT) content providers market in India with their data analytics and artificial intelligence platform. Speaking to ET, Bikram Bedi, the head of India region, Amazon Web Services said that globally, and here, media & entertainment has been a huge focus for the company. They work with OTT platforms or linear TV platforms like Hotstar, Voot (from Viacom18), Netflix, Amazon Prime, Sony, Zee TV, NDTV.com, The company's service called Midas from Elemental allows customers to monetise video offerings. Amazon Web Services will help OTT, media and entertainment players in the entire media life-chain right from downloading content to delivering it to customers. "So around analytics on video, we've launched a platform called Rekognition, which allows you to take video streams, store them in S3, and then analyse.

Most of the discussions around artificial intelligence, machine learning and intelligent automation tends to occur under the conceit that the changes anticipated by their adoption lie off a few years away as part of some future scenario that we really don't have to worry about just yet. The latest evidence that this is not true came this week from ServiceNow, the cloud software company that specializes in helping companies get simple things done. The company surveyed CIOs in 25 industries across 11 countries for its Global CIO Point of View and found that about half of them are already using some form of machine learning and another 40 percent have plans on the board to adopt it. That adds up to a total of 89 percent of CIOs who have already adopted machine learning or who say they're going to. I talked about the survey with Dave Wright, ServiceNow's chief strategy officer, and Chris Bedi, its CIO, who told me that among their customers, the talking phase is over, and the doing phase is underway.

ServiceNow uncovered more evidence that machine learning and other artificial intelligence (AI) technologies can help businesses run a tighter and potentially more profitable ship. The company's recent survey (PDF) of 500 CIOs from 11 countries, conducted by Oxford Economics, revealed that more than half (52 percent) of enterprises are using machine learning software to automate complex decision making after having already used the technology to automate routine tasks. A large majority (87 percent) expect to derive value out of those decisions. Sixty-nine percent believe that machine learning systems can make decisions that are more accurate than humans. "We see three kinds of processes as targets for machine learning--anything requiring rating, ranking or forecasting," said Chris Bedi, CIO at ServiceNow, in a statement.

A survey of 500 CIOs from around the world by ServiceNow finds that machine learning has arrived in the enterprise, making material contributions to everyday work. To realize its full value, technology leaders must find skilled talent to work side-by-side with machines, in addition to redesigning their organizations and processes. Go from Pro to Superhero! Our Amazing, Incredible, Invincible PR Strategy Checklist is jam-packed with wisdom and resources to keep you saving the day! For The Global CIO Point of View, ServiceNow surveyed CIOs in 11 countries across 25 industries to uncover the competitive benefits of adopting machine learning and hear how those leaders are driving results.

If you're concerned (or super excited) about machine learning (ML) becoming mainstream, a recent survey by Oxford Economics on behalf of human resources (HR) and IT asset management company ServiceNow should pique your interest. The report, which surveyed 500 Chief Information Officers (CIOs) in 11 countries and across 25 industries, found that 49 percent of the companies are already using ML to improve traditional business processes. Of the 500 CIOs surveyed, 200 said they're already beyond the pilot stage and have begun deploying ML in some capacity. CIOs are hoping to limit user error and errors in judgement by introducing automation. Almost 70 percent of CIOs said decisions made by machines will be more accurate than those made by humans.