Deep Learning
Beyond One-hot Encoding: lower dimensional target embedding
Rodríguez, Pau, Bautista, Miguel A., Gonzàlez, Jordi, Escalera, Sergio
Target encoding plays a central role when learning Convolutional Neural Networks. In this realm, One-hot encoding is the most prevalent strategy due to its simplicity. However, this so widespread encoding schema assumes a flat label space, thus ignoring rich relationships existing among labels that can be exploited during training. In large-scale datasets, data does not span the full label space, but instead lies in a low-dimensional output manifold. Following this observation, we embed the targets into a low-dimensional space, drastically improving convergence speed while preserving accuracy. Our contribution is two fold: (i) We show that random projections of the label space are a valid tool to find such lower dimensional embeddings, boosting dramatically convergence rates at zero computational cost; and (ii) we propose a normalized eigenrepresentation of the class manifold that encodes the targets with minimal information loss, improving the accuracy of random projections encoding while enjoying the same convergence rates. Experiments on CIFAR-100, CUB200-2011, Imagenet, and MIT Places demonstrate that the proposed approach drastically improves convergence speed while reaching very competitive accuracy rates.
IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
Espeholt, Lasse, Soyer, Hubert, Munos, Remi, Simonyan, Karen, Mnih, Volodymir, Ward, Tom, Doron, Yotam, Firoiu, Vlad, Harley, Tim, Dunning, Iain, Legg, Shane, Kavukcuoglu, Koray
In this work we aim to solve a large collection of tasks using a single reinforcement learning agent with a single set of parameters. A key challenge is to handle the increased amount of data and extended training time. We have developed a new distributed agent IMPALA (Importance Weighted Actor-Learner Architecture) that not only uses resources more efficiently in singlemachine training but also scales to thousands of machines without sacrificing data efficiency or resource utilisation. We achieve stable learning at high throughput by combining decoupled acting and learning with a novel off-policy correction method called V-trace. We demonstrate the effectiveness of IMPALA for multi-task reinforcement learning on DMLab-30 (a set of 30 tasks from the DeepMind Lab environment (Beattie et al., 2016)) and Atari-57 (all available Atari games in Arcade Learning Environment (Bellemare et al., 2013a)). Our results show that IMPALA is able to achieve better performance than previous agents with less data, and crucially exhibits positive transfer between tasks as a result of its multi-task approach. The source code is publicly available at github.com/deepmind/scalable
Multimedia Semantic Integrity Assessment Using Joint Embedding Of Images And Text
Jaiswal, Ayush, Sabir, Ekraam, AbdAlmageed, Wael, Natarajan, Premkumar
Real world multimedia data is often composed of multiple modalities such as an image or a video with associated text (e.g. captions, user comments, etc.) and metadata. Such multimodal data packages are prone to manipulations, where a subset of these modalities can be altered to misrepresent or repurpose data packages, with possible malicious intent. It is, therefore, important to develop methods to assess or verify the integrity of these multimedia packages. Using computer vision and natural language processing methods to directly compare the image (or video) and the associated caption to verify the integrity of a media package is only possible for a limited set of objects and scenes. In this paper, we present a novel deep learning-based approach for assessing the semantic integrity of multimedia packages containing images and captions, using a reference set of multimedia packages. We construct a joint embedding of images and captions with deep multimodal representation learning on the reference dataset in a framework that also provides image-caption consistency scores (ICCSs). The integrity of query media packages is assessed as the inlierness of the query ICCSs with respect to the reference dataset. We present the MultimodAl Information Manipulation dataset (MAIM), a new dataset of media packages from Flickr, which we make available to the research community. We use both the newly created dataset as well as Flickr30K and MS COCO datasets to quantitatively evaluate our proposed approach. The reference dataset does not contain unmanipulated versions of tampered query packages. Our method is able to achieve F1 scores of 0.75, 0.89 and 0.94 on MAIM, Flickr30K and MS COCO, respectively, for detecting semantically incoherent media packages.
How the UK can become a leader in artificial intelligence
From the Alan Turing Institute to DeepMind, the UK boasts a rich history and exciting present in machine learning and artificial intelligence research and development, led by academia and industry. According to recent research, AI is the largest commercial opportunity for Britain, projected to add £232 billion to the UK economy by 2030. SMEs and start-ups will play a significant role in grasping this. The segment highlighted by Theresa May during her speech at the World Economic Forum in Davos in January, where she told world leaders that the UK's strong start-up scene will be instrumental in making the UK a world leader in ethical AI. However, start-ups today still need to overcome some significant challenges before reaching their full potential.
Personalized 'deep learning' equips robots for autism therapy: Machine learning network offers personalized estimates of children's behavior
This type of therapy works best, however, if the robot can smoothly interpret the child's own behavior -- whether he or she is interested and excited or paying attention -- during the therapy. Researchers at the MIT Media Lab have now developed a type of personalized machine learning that helps robots estimate the engagement and interest of each child during these interactions, using data that are unique to that child. Armed with this personalized "deep learning" network, the robots' perception of the children's responses agreed with assessments by human experts, with a correlation score of 60 percent, the scientists report June 27 in Science Robotics. It can be challenging for human observers to reach high levels of agreement about a child's engagement and behavior. Their correlation scores are usually between 50 and 55 percent.
NervanaSystems/nlp-architect
NLP Architect is an open-source Python library for exploring the state-of-the-art deep learning topologies and techniques for natural language processing and natural language understanding. It is intended to be a platform for future research and collaboration. The library consists of core modules (topologies), data pipelines, utilities and end-to-end model examples with training and inference scripts. We look at these as a set of building blocks that were needed for implementing NLP use cases based on our pragmatic research experience. Each of the models includes algorithm descriptions and results in the documentation.
Drive Disruptive Innovation at Scale with AI NetApp Webcast
With the whirlwind pace of artificial intelligence (AI) and deep learning technology, many enterprises are challenged with how to advance new AI projects from proof of concept to production. Join the webcast "Drive Disruptive Innovation at Scale with AI" and find out how you can enable a secure and smooth flow of data for your AI workflows, from edge to core to cloud.
The Role of AI in Healthcare Technology – Becoming Human: Artificial Intelligence Magazine
Big data and its component, AI, are taking many industries by storm. The benefits of big data are far too significant for virtually any industry to ignore. Perhaps the industry that is improving most because of big data and AI is the healthcare industry. As soon as the healthcare industry realized the potential of implementing big data, it spread across all sectors like wildfire. AI has already established itself in healthcare in many ways, but healthcare professionals have only scratched the surface.
Are autonomous data centers on the horizon?
At some point in the not-too-distant future, artificial intelligence (AI) will drive our cars, write our programming code, and optimize how we do business. Data centers, too, will be unable to escape this trend. Thanks to machine learning technology, companies and data center operators will be able to coordinate and manage increasingly complex machines, infrastructures, and data more effectively than ever before, even as their numbers and data volumes continue to rise. Are completely autonomous, self-repairing data centers on the horizon? The data center is the backbone of the digital revolution.
A Bot Backed by Elon Musk Has Made an AI Breakthrough in Video Game World
Artificial-intelligence research group OpenAI said it created software capable of beating teams of five skilled human players in the video game Dota 2, a milestone in computer science. The achievement puts San Francisco-based OpenAI, whose backers include billionaire Elon Musk, ahead of other artificial-intelligence researchers in developing software that can master complex games combining fast, real-time action, longer-term strategy, imperfect information and team play. The ability to learn these kinds of video games at human or super-human levels is important for the advancement of AI because they more closely approximate the uncertainties and complexity of the real world than games such as chess, which IBM's software mastered in the late 1990s, or Go, which was conquered in 2016 with software created by DeepMind, the London-based AI company owned by Alphabet Inc. Dota 2 is a multiplayer science-fiction fantasy video game created by Bellevue, Washington-based Valve Corp. Each team is assigned a base on opposing ends of a map that can only be learned through exploration. Each player controls a separate character with unique powers and weapons. Each team must battle to reach the opposing team's territory and destroy a structure called an Ancient.