Deep Learning
Pixel Deconvolutional Networks
Gao, Hongyang, Yuan, Hao, Wang, Zhengyang, Ji, Shuiwang
Deconvolutional layers have been widely used in a variety of deep models for up-sampling, including encoder-decoder networks for semantic segmentation and deep generative models for unsupervised learning. One of the key limitations of deconvolutional operations is that they result in the so-called checkerboard problem. This is caused by the fact that no direct relationship exists among adjacent pixels on the output feature map. To address this problem, we propose the pixel deconvolutional layer (PixelDCL) to establish direct relationships among adjacent pixels on the up-sampled feature map. Our method is based on a fresh interpretation of the regular deconvolution operation. The resulting PixelDCL can be used to replace any deconvolutional layer in a plug-and-play manner without compromising the fully trainable capabilities of original models. The proposed PixelDCL may result in slight decrease in efficiency, but this can be overcome by an implementation trick. Experimental results on semantic segmentation demonstrate that PixelDCL can consider spatial features such as edges and shapes and yields more accurate segmentation outputs than deconvolutional layers. When used in image generation tasks, our PixelDCL can largely overcome the checkerboard problem suffered by regular deconvolution operations.
Knowledge Graph Completion via Complex Tensor Factorization
Trouillon, Thรฉo, Dance, Christopher R., Welbl, Johannes, Riedel, Sebastian, Gaussier, รric, Bouchard, Guillaume
In statistical relational learning, knowledge graph completion deals with automatically understanding the structure of large knowledge graphs---labeled directed graphs---and predicting missing relationships---labeled edges. State-of-the-art embedding models propose different trade-offs between modeling expressiveness, and time and space complexity. We reconcile both expressiveness and complexity through the use of complex-valued embeddings and explore the link between such complex-valued embeddings and unitary diagonalization. We corroborate our approach theoretically and show that all real square matrices---thus all possible relation/adjacency matrices---are the real part of some unitarily diagonalizable matrix. This results opens the door to a lot of other applications of square matrices factorization. Our approach based on complex embeddings is arguably simple, as it only involves a Hermitian dot product, the complex counterpart of the standard dot product between real vectors, whereas other methods resort to more and more complicated composition functions to increase their expressiveness. The proposed complex embeddings are scalable to large data sets as it remains linear in both space and time, while consistently outperforming alternative approaches on standard link prediction benchmarks.
Deep learning and big data: Wall Street and the new data paradigm
Wall Street is big business, and it is about to become even bigger with the rise of big data. It is every investor's dream to have prior knowledge of the direction of the market before it happens, which is why financial investment firms are driven to mine for data rather than for gold in the information economy. Traditionally, investors have based their decisions on fundamentals, intuition, and analysis drawn from traditional data sources, such as quarterly earnings reports, financial statement filings to the U.S. Securities and Exchange Commission (SEC), historical market data, institutional research reports and sometimes the so-called "expert networks." The new data-driven paradigm, fueled by new alternative data sources, high performance computing and predictive analytics, offers a more robust framework to generate data-driven investment theses. Data โ from satellite images of areas of interest, automated drones, people-counting sensors, container ships' positions, credit card transactional data, jobs and layoffs reports, cell phones, social media, news articles, tweets, online search queries โ is now the most valuable commodity for Wall Street.
Counting Crowds and Lines ยท dimroc
In Union Square, NYC, there's the untoppable burger joint name Shake Shack that's always crowded. A group of us would obsessively check the Shake Cam around lunch to figure out if that trip was worth it. Rather than do this manually (come on, it's nearly 2018), it would be great if this could be done for us. Then, to take that idea further, imagine being able to measure foot traffic on a month to month basis or to measure the impact of a new promotional campaign. Object detection has received a lot of attention in the deep learning space, but it's ill-suited for highly congested scenes like crowds.
AWS ramps up in AI with new consultancy services and Rekognition features
Ahead of Amazon's big AWS division Re:invent conference next week, the company has announced two developments in the area of artificial intelligence. AWS is opening a machine learning lab, ML Solutions Lab, to pair Amazon machine learning experts with customers looking to build solutions using the AI tech. And it's releasing new features within Amazon Rekognition, Amazon's deep learning-based image recognition platform: real-time face recognition and the ability to recognize text in images. The new lab and the enhancements to its image recognition platform underscore the push that Amazon and AWS are giving to AI at the company, both internally and as a potential area to grow its B2B business in this area. They come about a month after AWS announced it would be collaborating with Microsoft on Gluon, a deep learning interface designed for developers to build and run machine learning models for their apps and other services.
An AI Resident at work: Suhani Vora and her work on genomics
Suhani: During graduate school, I worked on engineering CRISPR/Cas9 systems, which enable a wide range of research on genomes. And though I was working with the most efficient tools available for genome editing, I knew we could make progress even faster. One important factor was our limited ability to predict what novel biological designs would work. Each design cycle, we were only using very small amounts of previously collected data and relied on individual interpretation of that data to make design decisions in the lab. By failing to incorporate more powerful computational methods to make use of big data and aid in the design process, it was affecting our ability to make progress quickly.
Speeding up DQN on PyTorch: how to solve Pong in 30 minutes
Some time ago I've implemented all models from the article Rainbow: Combining Improvements in Deep Reinforcement Learning using PyTorch and my small RL library called PTAN. The code of eight systems is here if you're curious. To debug and test it I've used Pong game from Atari suite, mostly due to its simplicity, fast convergence, and hyperparameters robustness: you can use from 10 to 100 smaller size of replay buffer and it still will converge nicely. This is extremely helpful for a Deep RL enthusiast without access to the computational resources Google employees have. During implementation and debugging of the code, I was needed to run about 100โ200 optimisations, so, it does matter how long one run takes: 2โ3 days or just an hour. Nevertheless you always should keep a balance here: trying to squeeze as much performance as possible, you can introduce bugs, which will dramatically complicate already complex debugging and implementation process.
New Cray Artificial Intelligence Initiatives to Advance Deep Learning for Science and Enterprise - insideBIGDATA
Cray Inc. (Nasdaq:CRAY) announced a comprehensive set of Artificial Intelligence (AI) products and programs that will empower customers to learn, start, and scale their deep learning initiatives. As AI and deep learning continue to transform entire industries and scientific disciplines, Cray is leveraging its supercomputing expertise, technologies, and best practices to advance the adoption of deep learning. An AI collaboration agreement with Intel, leveraging Intel's AI technologies to advance the state-of-the-art in distributed deep learning and machine learning. Cray is committed to working closely with our customers, partners, and innovators in AI to drive the adoption of deep learning in science and enterprise," said Fred Kohout, Cray's senior vice president of products and chief marketing officer. "At Cray, we are bringing together a powerful set of innovative systems, software, deep learning architectures, and a hands-on lab environment to give organizations a trusted partner ...
Dell EMC launches new machine and deep learning solutions
Dell EMC announced the launch of its new machine learning and deep learning solutions, which according to the company is in line with it continuing its work to bring high-performance computing (HPC) and data analytics capabilities to mainstream enterprises worldwide. Dell EMC believes that this enables organisations to take advantage of the convergence of HPC and data analytics and realise advancements in areas including fraud detection, image processing, financial investment analysis and personalised medicine. According to the company, these new innovations represent the next step in the company's focus on democratising HPC, optimising data analytics with artificial intelligence (AI) technology innovations, and advancing both the HPC and AI communities. While AI techniques, such as machine learning and deep learning, being rapidly being deployed by many organisations across several industries, only a small number possess the expertise to design, deploy and manage such systems to use them effectively for rapidly gaining new insights. Dell EMC believes that by leveraging Dell's ecosystem of partnerships and internal expertise in HPC and data analytics services, the company's new solutions offer customers the ability to harness the power of the massive amounts of their collected data, delivering faster, better and deeper business insights in real-time.
Capsule Networks Are Shaking up AI -- Here's How to Use Them
If you follow AI you might have heard about the advent of the potentially revolutionary Capsule Networks. I will show you how you can start using them today. Geoffrey Hinton is known as the father of "deep learning." Back in the 50s the idea of deep neural networks began to surface and, in theory, could solve a vast amount of problems. However, nobody was able to figure out how to train them and people started to give up.