When dealing with machine learning problems, sometimes one has to face a huge dataset with hundreds or thousands of features. Machine learning relies on these data to build models for prediction, more information the features contain, more easier to train a good model. However, these variables also contain noise, and most of them might be anonymous or formatted by some kind of hash process due to privacy issue or confidential reasons. So it's hard to figure out the physical meaning and explain the correlation between these variables. Even we could know all variables' meanings, it's still difficult to determine which are more essential than others.

Search on mobile can be a major pain. To start, the keyboard is small. Then there are the load times. Have you ever stared at your 4G smartphone screen and wondered in disbelief how only one bar of 3G was possible in San Francisco – in the country that put the first man on the Moon? How often have you longed for your laptop to be able to "search properly"?

Over July 4th weekend in 1981, several hundred game nerds gathered at a banquet hall in San Mateo, California. Personal computing was still in its infancy, and the tournament was decidedly low-tech. Each match played out on a rectangular table filled with paper game pieces, and a March Madness-style tournament bracket hung on the wall. The game was called Traveller Trillion Credit Squadron, a role-playing pastime of baroque complexity. Contestants did battle using vast fleets of imaginary warships, each player guided by an equally imaginary trillion-dollar budget and a set of rules that spanned several printed volumes. If they won, they advanced to the next round of war games--until only one fleet remained. Doug Lenat, then a 29-year-old computer science professor at nearby Stanford University, was among the players. But he didn't compete alone. He entered the tournament alongside Eurisko, the artificially intelligent system he built as part of his academic research. Eurisko ran on dozens of machines inside Xerox PARC--the computer research lab just down the road from Stanford that gave rise to the graphical user interface, the laser printer, and so many other technologies that would come to define the future of computing. That year, Lenat taught Eurisko to play Traveller. Doug Lenat says his common-sense engine is a new dawn for AI. The rest of the tech world doesn't really agree with him. Lenat fed the massive Traveller rulebook into the system and asked it to find the best way of winning.

Since announcements late last year about Google open-sourcing TensorFlow, the company's open-source library for machine learning, and previous coverage at InfoQ, the data-science community has had an opportunity to try out TensorFlow for their own projects. Databricks' Tim Hunter demonstrates TensorFlow-generated model selection and at-scale neural network processing with Spark. Hunter describes an artificial neural network as mimicking the neurons in the visual cortex of the human brain, which when adequately trained can be used for processing complex input data like imagery or audio. Hunter detailed how he ran TensorFlow on various Spark configurations to parallelize hyperparameter tuning. Hunter stated that TensorFlow, currently available with Python and C support helped "automate the creation of training algorithms for neural networks of various shapes and sizes" for the purpose of training a neural network to process large amounts of data with high accuracy and optimal runtime performance.

With the Watson visual-recognition service, you can analyse images or video frames to understand their contents. Image recognition is a hot area of research using artificial intelligence, and now IBM offers an online demo to let anyone test out the capabilities offered by its Watson cognitive computing system. Six sample photos are provided for illustration, or you can upload your own and ask Watson to analyze them. Either way, the cognitive system will produce a series of "classifiers" offering descriptions of the image's contents along with confidence scores for each of them. You can also create custom classifiers tailored for specific purposes.

As R programming language becoming popular more and more among data science group, industries, researchers, companies embracing R, going forward I will be writing posts on learning Data science using R. The tutorial course will include topics on data types of R, handling data using R, probability theory, Machine Learning, Supervised – unSupervised learning, Data Visualization using R, etc. Before going further, let's just see some stats and tidbits on data science and R.

I've been doing a little bit of reading on optimization (from Nocedal's book) and have some questions about the prevalence of SGD and variants such as Adam for training neural nets. L-BFGS and other quasi-Newton methods have both theoretical and experimentally verified (PDF) faster convergence. Are there any good reasons training with L-BFGS is much less popular (or at least talked about) than SGD and variants? For the deep learning practitioners, have you ever tried using L-BFGS or other quasi-Newton or conjugate gradient methods? In a similar vein, has anyone experimented with doing a line search for optimal step size during each gradient descent step?

A mesmerising simulation of the sun has provided the most accurate representation of its magnetic field to date. The hypnotic map shows the complex and turbulent forces that shapes the ball of plasma, causing violent solar flares and coronal mass ejections. Researchers hope the model could help solve a long-standing mystery about our star; if the magnetic lines on the surface are so chaotic, how is it that it has a unified magnetic field? A mesmerising simulation of the sun has provided the most accurate representation of its magnetic field to date. The hypnotic map shows the complex and turbulent forces that shapes the ball of plasma, causing violent solar flares and coronal mass ejections.

As many of you know by now, on Wednesday we launched a chatbot called Tay. We are deeply sorry for the unintended offensive and hurtful tweets from Tay, which do not represent who we are or what we stand for, nor how we designed Tay. Tay is now offline and we'll look to bring Tay back only when we are confident we can better anticipate malicious intent that conflicts with our principles and values. I want to share what we learned and how we're taking these lessons forward. For context, Tay was not the first artificial intelligence application we released into the online social world.

One of the things that set statistics apart from the rest of applied mathematics is an interest in the problems introduced by sampling: how can we learn about a model if we're given only a finite and potentially noisy sample of data? Although frequently important, the issues introduced by sampling can be a distraction when the core difficulties you face would persist even with access to an infinite supply of noiseless data. For example, if you're fitting a misspecified model \(m_1\) to data generated by a model \(m_2\), this misspecification will persist even as the supply of data becomes infinite. In this setting, the issues introduced by sampling can be irrelevant: it's often more important to know whether or not the misspecified model, \(m_1\), could ever act as an acceptable approximation to the true model, \(m_2\). Until recently, I knew very little about these sorts of issues.