We study the problem of online learning in a class of Markov decision processes known as linearly solvable MDPs. In the stationary version of this problem, a learner interacts with its environment by directly controlling the state transitions, attempting to balance a fixed state-dependent cost and a certain smooth cost penalizing extreme control inputs. In the current paper, we consider an online setting where the state costs may change arbitrarily between consecutive rounds, and the learner only observes the costs at the end of each respective round. We are interested in constructing algorithms for the learner that guarantee small regret against the best stationary control policy chosen in full knowledge of the cost sequence. Our main result is showing that the smoothness of the control cost enables the simple algorithm of following the leader to achieve a regret of order $\log^2 T$ after $T$ rounds, vastly improving on the best known regret bound of order $T^{3/4}$ for this setting.

To meet the growing demand for AI expertise, companies are offering online education courses to prepare the workforce for the future. Increasingly, computers and devices learn and act on their own using software algorithms, the building blocks for artificial intelligence (AI) and machine learning (ML). Getting smartphones to understand voice commands, smart home sprinkler systems to change with the weather and online services to predict what people want requires programmers skilled in AI and ML. Demand for these coding skills is skyrocketing. Making devices smart and proactive remains controversial to anyone who fears that automation will lead to human job loss.

The vast majority of optimization and online learning algorithms today require some prior information about the data (often in the form of bounds on gradients or on the optimal parameter value). When this information is not available, these algorithms require laborious manual tuning of various hyperparameters, motivating the search for algorithms that can adapt to the data with no prior information. We describe a frontier of new lower bounds on the performance of such algorithms, reflecting a tradeoff between a term that depends on the optimal parameter value and a term that depends on the gradients' rate of growth. Further, we construct a family of algorithms whose performance matches any desired point on this frontier, which no previous algorithm reaches.

As with learning any new skills, the more you practice, the better you become. Practice different algorithms and work with different data sets to have a better understanding of machine learning, and to improve your overall problem-solving skills. Machine learning with Python is a great addition to your technical skillset, and there are lots of free and low-cost online resources available to help. How have you picked up machine learning skills? Leave a comment below, or submit an article proposal to share your story.

Throughout my career, I have been investigating how new technology and research results can be of benefit for the average user. Over the years I worked with cutting edge technology (Artificial Intelligence, Semantic Web, Web 2.0, mobile applications) and investigated its potential to be employed in daily life, by non-experts. I have years of expertise in coordinating international teams. I like to talk about and present latest technology and research results to laymen, for instance at Barcamps and as an invited speaker at innovation fairs. Throughout my career, I have been investigating how new technology and research results can be of benefit for the average user.

If you find yourself daydreaming during class, Nestor could help you regain your focus and help professors improve the least-engaging parts of their lecture. You may have gotten away with staring off into space during your college years, but alas, the students of tomorrow may not be so lucky. In fact, some business students of today will soon find that their attention spans (or lack thereof) are being closely monitored. It's all thanks to a combination of artificial intelligence and facial analysis, which researchers are using to detect whether or not students are actually paying attention in lectures. This combination forms a new kind of software called Nestor, and at its September launch, will be used in two online courses at the ESG business school. It's the brainchild of LCA Learning, and it may just change the way we take classes.

That's because, to paraphrase Amazon's Jeff Bezos, artificial intelligence (AI) is "not just in the first inning of a long baseball game, but at the stage where the very first batter comes up." Look around, and you will find AI everywhere--in self driving cars, Siri on your phone, online customer support, movie recommendations on Netflix, fraud detection for your credit cards, etc. To be sure, there's more to come. Featuring 30 lectures, MIT's course "introduces students to the basic knowledge representation, problem solving, and learning methods of artificial intelligence." It includes interactive demonstrations designed to "help students gain intuition about how artificial intelligence methods work under a variety of circumstances."

IN JULY 2011 Sebastian Thrun, who among other things is a professor at Stanford, posted a short video on YouTube, announcing that he and a colleague, Peter Norvig, were making their "Introduction to Artificial Intelligence" course available free online. By the time the course began in October, 160,000 people in 190 countries had signed up for it. At the same time Andrew Ng, also a Stanford professor, made one of his courses, on machine learning, available free online, for which 100,000 people enrolled. Both courses ran for ten weeks. Such online courses, with short video lectures, discussion boards for students and systems to grade their coursework automatically, became known as Massive Open Online Courses (MOOCs).

Myself along with colleagues just published the Cool Vendors in Information Governance and MDM. Data and analytics leaders struggle to leverage data to drive innovation and govern their information assets effectively. New approaches suggest disruptive efforts to drive both innovation and effective governance will change the economics and complexity of innovation.

The explosion of data from high throughput biological experiments like sequencing and micro-arrays has led to the science called Bioinformatics. Bioinformatics is the interdisciplinary science which is similar to Data Science for solving biological problems. According to Wikipedia "Bioinformatics is an interdisciplinary science, ultimately aiming to understand biology". Our human body can be break down into small machineries of cells which is involved in complex processes. These cells are controlled by the central processing unit called DNA (De-oxyribo Nucleic Acid).