Many data science competitions suffer from a test set being markedly different from a training set (a violation of the "identically distributed" assumption). It is then difficult to make a representative validation set. We propose a method for selecting training examples most similar to test examples and using them as a validation set. The core of this idea is training a probabilistic classifier to distinguish train/test examples. In part one, we inspect the ideal case: training and testing examples coming from the same distribution, so that the validation error should give good estimation of the test error and classifier should generalize well to unseen test examples.

In the field of machine learning and specifically the problem of statistical classification, a confusion matrix, also known as an error matrix,[4] is a specific table layout that allows visualization of the performance of an algorithm, typically a supervised learning one (in unsupervised learning it is usually called a matching matrix). Each column of the matrix represents the instances in a predicted class while each row represents the instances in an actual class (or vice-versa).[2] The name stems from the fact that it makes it easy to see if the system is confusing two classes (i.e. It is a special kind of contingency table, with two dimensions ("actual" and "predicted"), and identical sets of "classes" in both dimensions (each combination of dimension and class is a variable in the contingency table). If a classification system has been trained to distinguish between cats, dogs and rabbits, a confusion matrix will summarize the results of testing the algorithm for further inspection.

As mentioned before, we like working with graphs because the mathematical construct inherently captures relationships that matter. But to move beyond theorems and proofs -- to effectively use graphs in the real world -- we need ways to store and analyze them within a team environment. So how does our Dendrite open source project address that challenge? In short, it ties together modified versions of leading open source technologies, adds a base capability for graph collaboration, and uses a web interface to drive it all. To understand how we got here, it helps to have a notion of how we work.

Exploding phones ensured Samsung lost out to Apple during the adjacent launches of their newest smartphones, and Samsung could be emphasizing big software upgrades to divert attention from the much-publicized failure of the Galaxy Note 7. Directly buying out the company that developed Siri, one of Apple's iconic mobile features, is indeed a bold competitive move, and Viv touts even better functionality than its parent tech. A key differentiation point for Viv is that it purports to be developer friendly. This is significant as apps for digital assistants will likely be a key method for driving use going forward. It appears that digital assistants are becoming table stakes for consumer-facing digital platforms, with Google, Microsoft, Amazon and Apple all offering one. This suggests that Samsung's acquisition is a strategic move to insure it stays with the pack.

Omron's table tennis robot is getting smarter. At this week's Ceatec electronics show in Japan, the company has unveiled a new version that uses machine learning to assess the strength of an opponent and ramp up its game accordingly. The robot, named Forpheus, was first shown at the event in 2014 to impressive reviews. A series of robotic arms manipulate a table tennis bat, guided by cameras that watch the ball and predict where it's going to land. Omron developed the robot to promote the company's sensor technology and this latest version is showing off some of what can be done when sensing combines with machine learning and artificial intelligence.

Nasa has announced that it has found evidence of flowing water on Mars. Scientists have long speculated that Recurring Slope Lineae -- or dark patches -- on Mars were made up of briny water but the new findings prove that those patches are caused by liquid water, which it has established by finding hydrated salts. Several hundred camped outside the London store in Covent Garden. The 6s will have new features like a vastly improved camera and a pressure-sensitive "3D Touch" display

Out of sight, out of mind – overfishing is one of the biggest environment issues facing us today, with over 85 percent of the world's fisheries in dire need of protection but then again what happens beyond the horizon in the middle of oceans is hardly monitored or known to the public. Google, in partnership with Oceana and SkyTruth, has launched a tracking platform called "Global Fishing Watch" to help increase awareness of fisheries and to monitor fishing activities across the world. Vessels in the ocean use the Automatic Identification System (AIS) to transmit their location for identifying themselves and for communicating with nearby ships, AIS base stations and satellites. At any given time, there are approximately 200,000 ships that broadcast their location using the AIS. Global Fishing Watch utilises machine learning to comb through these logs (over 22 million records per day) to classify and determine the type of ship (e.g., cargo, tug, sail, fishing) and the kind of fishing gear (longline, purse seine, trawl) they're using.

Artificial intelligence has been a trope of science fiction for decades. From the unsettling and double-crossing HAL in the film 2001: A Space Odyssey to the oppressive robots enslaving humanity in The Matrix and The Terminator, the concept of sentient machines outsmarting and overthrowing their creators has long permeated popular culture. If these movies are anything to go by, credit risk managers should be worried. Machine learning, a subset of AI that allows computers to answer simple questions...

The idea of autonomous robotic cars have always held a special place in sci-fi culture. Technology is now catching up. In 2009 many luxury brands began introducing assisted cruise control and adaptive lane change software built with endless reams of modelling data. More recently Tesla used big data and artificial intelligence to launch its Autopilot feature. Nvidia and Alphabet use artificial intelligence to build detailed real-time maps their test vehicles use to see the world.

Human (or any other animal for that matter) brain computational power is limited by two basic evolution requirements: survival and procreation. Our "hardware" (physiology) and "software" (hard-coded nature psychology) only had to evolve to allow us to perform a set of basic actions - identify Friend or Foe, obtain food, find our place in the social tribe hierarchy, ultimately find a mate and multiply. Anything beyond this point, or not directly leading to this point can be considered redundant, when viewed from the evolution perspective. To accomplish these "life" goals, our brains evolved to a certain physical limit (100 billion neurons per average brain, on average 7000 synaptic connections per neuron). Obviously, evolving beyond this limit was not beneficiary for survival and procreation in the African savannas.