Performance-critical machine learning models should be robust to input perturbations not seen during training. Adversarial training is a method for improving a model's robustness to some perturbations by including them in the training process, but this tends to exacerbate other vulnerabilities of the model. The adversarial training framework has the effect of translating the data with respect to the cost function, while weight decay has a scaling effect. Although weight decay could be considered a crude regularization technique, it appears superior to adversarial training as it remains stable over a broader range of regimes and reduces all generalization errors. Equipped with these abstractions, we provide key baseline results and methodology for characterizing robustness. The two approaches can be combined to yield one small model that demonstrates good robustness to several white-box attacks associated with different metrics.

We construct a multilingual common semantic space based on distributional semantics, where words from multiple languages are projected into a shared space to enable knowledge and resource transfer across languages. Beyond word alignment, we introduce multiple cluster-level alignments and enforce the word clusters to be consistently distributed across multiple languages. We exploit three signals for clustering: (1) neighbor words in the monolingual word embedding space; (2) character-level information; and (3) linguistic properties (e.g., apposition, locative suffix) derived from linguistic structure knowledge bases available for thousands of languages. We introduce a new cluster-consistent correlational neural network to construct the common semantic space by aligning words as well as clusters. Intrinsic evaluation on monolingual and multilingual QVEC tasks shows our approach achieves significantly higher correlation with linguistic features than state-of-the-art multi-lingual embedding learning methods do. Using low-resource language name tagging as a case study for extrinsic evaluation, our approach achieves up to 24.5\% absolute F-score gain over the state of the art.

Image segmentation needs both local boundary position information and global object context information. The performance of the recent state-of-the-art method, fully convolutional networks, reaches a bottleneck due to the neural network limit after balancing between the two types of information simultaneously in an end-to-end training style. To overcome this problem, we divide the semantic image segmentation into temporal subtasks. First, we find a possible pixel position of some object boundary; then trace the boundary at steps within a limited length until the whole object is outlined. We present the first deep reinforcement learning approach to semantic image segmentation, called DeepOutline, which outperforms other algorithms in Coco detection leaderboard in the middle and large size person category in Coco val2017 dataset. Meanwhile, it provides an insight into a divide and conquer way by reinforcement learning on computer vision problems.

Person re-identification (re-id) suffers from a serious occlusion problem when applied to crowded public places. In this paper, we propose to retrieve a full-body person image by using a person image with occlusions. This differs significantly from the conventional person re-id problem where it is assumed that person images are detected without any occlusion. We thus call this new problem the occluded person re-identitification. To address this new problem, we propose a novel Attention Framework of Person Body (AFPB) based on deep learning, consisting of 1) an Occlusion Simulator (OS) which automatically generates artificial occlusions for full-body person images, and 2) multi-task losses that force the neural network not only to discriminate a person's identity but also to determine whether a sample is from the occluded data distribution or the full-body data distribution. Experiments on a new occluded person re-id dataset and three existing benchmarks modified to include full-body person images and occluded person images show the superiority of the proposed method.

One of the poorest-kept secrets in Silicon Valley has been the huge salaries and bonuses that experts in artificial intelligence can command. Now, a little-noticed tax filing by a research lab called OpenAI has made some of those eye-popping figures public. OpenAI paid its top researcher, Ilya Sutskever, more than $1.9 million in 2016. It paid another leading researcher, Ian Goodfellow, more than $800,000 -- even though he was not hired until March of that year. Both were recruited from Google.

Differences in imaging equipment, procedures and protocols can dramatically affect the performance of deep machine learning when analyzing brain tumors, according to a new study in Medical Physics. Automatic brain tumor segmentation from MRI data using deep learning methodologies has gained steam in recent years. Convolutional neural networks (CNNs), a type of deep learning algorithm, are commonly used for segmentation of brain tumors, and provider organizations have recently begun sharing images to increase the data to work with. However, providers often use different imaging equipment, image acquisition parameters and contrast injection protocols, which could cause institutional bias; a CNN model trained on MRI data from one organization may stumble when tested on MRI data from another. The researchers, from the Radiology Department at Duke University School of Medicine, used MRI data of 22 glioblastoma patients from MD Anderson Cancer Center and 22 glioblastoma patients from Henry Ford Hospital to assess how CNN models worked with their own and each other's MRI data.

Most of us have used apps like Shazam, which can identify songs when we hold up our phone up to a speaker. But what if it was possible for an app to identify a piece of music based on nothing more than your thought patterns. Perhaps not, according to a new piece of research carried out by investigators at the University of California, Berkeley. In 2014, researcher Brian Pasley and colleagues used a deep-learning algorithm and brain activity, measured with electrodes, to turn a person's thoughts into digitally synthesized speech. This was achieved by analyzing a person's brain waves while they were speaking in order to decode the link between speech and brain activity.

Welcome to this course: Using Deep Learning To Make Decisions. Deep Learning a trending topic in the field of Artificial Intelligence today and can be considered to be an advanced form of machine learning, which is quite tricky to master. Deep learning is synonymous with machine learning, and simply an advanced subset of that larger field. Technically speaking, deep learning is an umbrella term for a set of neural nets that consist of three or more layers; i.e. at least one hidden layer, and the visible layers of input and output. So what is deep learning capable and incapable of?

Google's AlphaGo made history in May 2017 when it defeated Ke Jie, the world's reigning champion of the ancient Chinese game Go. It was the first computer program to defeat a professional human Go player, much less a world champion. Later that year, Google introduced AlphaGo Zero, an even more powerful iteration of AlphaGo. Anyone wanting to understand the difference between artificial intelligence and deep learning can start by understanding the difference between AlphaGo and AlphaGo Zero. With AlphaGo, Google trained the original AlphaGo to play by teaching it to look at data from the top players, said Avi Reichental, CEO of XponentialWorks.

Deep learning is not a beginner-friendly subject -- even for experienced software engineers and data scientists. If you've been Googling this subject, you may have been confused by the resources you've come across. To find the best resources, we surveyed engineers on their favorite sources for deep learning, and these are what they recommended. These educational resources include online courses, in-person courses, books, and videos. All are completely free and designed by leading professors, researchers, and industry professionals like Geoffrey Hinton, Yoshua Bengio, and Sebastian Thrun.