The Information Bottleneck (IB) is a conceptual method for extracting the most compact, yet informative, representation of a set of variables, with respect to the target. It generalizes the notion of minimal sufficient statistics from classical parametric statistics to a broader information-theoretic sense. The IB curve defines the optimal trade-off between representation complexity and its predictive power. Specifically, it is achieved by minimizing the level of mutual information (MI) between the representation and the original variables, subject to a minimal level of MI between the representation and the target. This problem is shown to be in general NP hard. One important exception is the multivariate Gaussian case, for which the Gaussian IB (GIB) is known to obtain an analytical closed form solution, similar to Canonical Correlation Analysis (CCA). In this work we introduce a Gaussian lower bound to the IB curve; we find an embedding of the data which maximizes its "Gaussian part", on which we apply the GIB. This embedding provides an efficient (and practical) representation of any arbitrary data-set (in the IB sense), which in addition holds the favorable properties of a Gaussian distribution. Importantly, we show that the optimal Gaussian embedding is bounded from above by non-linear CCA. This allows a fundamental limit for our ability to Gaussianize arbitrary data-sets and solve complex problems by linear methods.

In machine learning and optimization, one often wants to minimize a convex objective function $F$ but can only evaluate a noisy approximation $\hat{F}$ to it. Even though $F$ is convex, the noise may render $\hat{F}$ nonconvex, making the task of minimizing $F$ intractable in general. As a consequence, several works in theoretical computer science, machine learning and optimization have focused on coming up with polynomial time algorithms to minimize $F$ under conditions on the noise $F(x)-\hat{F}(x)$ such as its uniform-boundedness, or on $F$ such as strong convexity. However, in many applications of interest, these conditions do not hold. Here we show that, if the noise has magnitude $\alpha F(x) + \beta$ for some $\alpha, \beta > 0$, then there is a polynomial time algorithm to find an approximate minimizer of $F$. In particular, our result allows for unbounded noise and generalizes those of Applegate and Kannan, and Zhang, Liang and Charikar, who proved similar results for the bounded noise case, and that of Belloni et al. who assume that the noise grows in a very specific manner and that $F$ is strongly convex. Turning our result on its head, one may also view our algorithm as minimizing a nonconvex function $\hat{F}$ that is promised to be related to a convex function $F$ as above. Our algorithm is a "simulated annealing" modification of the stochastic gradient Langevin Markov chain and gradually decreases the temperature of the chain to approach the global minimizer. Analyzing such an algorithm for the unbounded noise model and a general convex function turns out to be challenging and requires several technical ideas that might be of independent interest in deriving non-asymptotic bounds for other simulated annealing based algorithms.

Tokyo, Japan may have just become the first city to officially grant residence to an artificial intelligence (AI). The intelligence's name is Shibuya Mirai and exists only as a chatbot on the popular Line messaging app. Mirai, which translates to'future' from Japanese, joins Hanson Robotic's "Sophia" as pioneering AI gaining statuses previously reserved for living, biological entities. The Kingdom of Saudi Arabia granted Sophia citizenship last month. The Shibuya Ward of Tokyo released a statement through Microsoft saying, "His hobbies are taking pictures and observing people. And he loves talking with people… Please talk to him about anything."

Trump opens Asia trip with Japan's Abe against backdrop of tensions with North Korea Just one in three Americans trust Trump to handle North Korean tensions well Japan's Abe treats Trump to a day of personal diplomacy, including golf and trucker hats Brazile says Democratic primaries weren't'rigged' though some see evidence in her new book Trump is silent on Saudi king's purge though he and Salman spoke by phone Japan's Abe treats Trump to a day of personal diplomacy, including golf and trucker hats Brazile says Democratic primaries weren't'rigged' though some see evidence in her new book Trump is silent on Saudi king's purge though he and Salman spoke by phone The greatest benefit from the House Republican tax bill would go to upper-income households, according to an analysis released Monday by the nonpartisan Tax Policy Center. Middle-income taxpayers -- those earning between $48,600 and $86,100 annually -- would receive an average tax cut of $700 next year, or about 1% of their after-tax income, the analysis said. The top 20% of the nation's earners -- those making more than $149,400 a year -- would receive an average tax cut of $4,850, or about 1.4% of after-tax income. Those top earners would also receive 60% of the total tax benefits under the plan. Of that, the top 1% of earners, defined as those making more than $730,000 a year, receive about 22% of the total amount of tax cuts in 2018, the Tax Policy Center said.

Participants and startups in the emerging self-driving vehicles industry (components, systems, trucks, cars and buses) have been at it for over almost 60 years. The pace accelerated in 2004, 2005 and 2007 when DARPA sponsored long-distance competitions for driverless cars, and then again in 2009 when Uber began its ride-hailing system. As the prospects that self-driving ride-hailing fleets, vehicles, systems and associated AI would soon be a reality, startups, fundings, mergers and acquisitions have followed reaching a peak in 2017. Thus far in 2017 more than 55 companies and startups offering everything from solid state distancing sensors to ride-share fleets and mapping systems – plus five strategic acquisitions – raised over $28.2 billion! Listed below are month-by-month recaps of self-driving-related fundings and acquisitions as reported by The Robot Report.

An artificial intelligence character has been made an official resident of a busy central Tokyo district. The virtual newcomer, named'Shibuya Mirai' resembles a chatty seven-year-old boy. And while the character does not exist physically, he can have text conversations with humans on the widely used LINE messaging app. An artificial intelligence character has been made an official resident of a busy central Tokyo district. The virtual newcomer, named'Shibuya Mirai' resembles a chatty seven-year-old boy The virtual newcomer, named'Shibuya Mirai' resembles a chatty seven-year-old boy.

A single Sophia with the right to vote, serve on juries, and win elections will have little to no substantive effect in this country, particularly since Hanson and Saudi Arabia have publicly stated she will remain in Saudi Arabia in a planned city where robots are expected to outnumber people. But if Hanson can make one Sophia, the company and others like it can make 100 Sophias, or 1,000 Sophias, or 1 million Sophias. Although Saudi Arabia may be unlikely to use this robot citizenship as a publicity ploy more than once, other countries have shown they will grant special favors for foreign investment. If a company were interested in gaining broad access to the rights of U.S. citizenship, it seems likely that there's at least one nation out there that will trade thousands or millions of robot citizens for manufacturing plants or other economic activity in its borders. If all those robots obtain citizenship in the United States, they could be the next big voter demographic.

Cutting Edge Deep Learning For Coders, Part 2 from fast.ai

A StarCraft player managed to outsmart several artificial intelligence bots in a recent tournament. Song Byung-gu, from South Korea, beat four AI players in the first StarCraft competition to pit real and digital gamers against each other. The multiple victories came at Sejong University in Seoul, which hosts an annual StarCraft AI contest. One of the bots beaten by Song was CherryPi, a software developed by Facebook's AI research team. Song beat all four bots within 27 minutes, with the shortest match lasting just four and a half minutes.

Subscribe to the O'Reilly Data Show Podcast to explore the opportunities and techniques driving big data and data science: Stitcher, TuneIn, iTunes, SoundCloud, RSS. In this episode of the O'Reilly Data Show, I spoke with Fang Yu, co-founder and CTO of DataVisor. We discussed her days as a researcher at Microsoft, the application of data science and distributed computing to security, and hiring and training data scientists and engineers for the security domain. DataVisor is a startup that uses data science and big data to detect fraud and malicious users across many different application domains in the U.S. and China. Founded by security researchers from Microsoft, the startup has developed large-scale unsupervised algorithms on top of Apache Spark, to (as Yu notes in our chat) "predict attack vectors early among billions of users and trillions of events."