We propose a method based on deep learning to perform cardiac segmentation on short axis MRI image stacks iteratively from the top slice (around the base) to the bottom slice (around the apex). At each iteration, a novel variant of U-net is applied to propagate the segmentation of a slice to the adjacent slice below it. In other words, the prediction of a segmentation of a slice is dependent upon the already existing segmentation of an adjacent slice. 3D-consistency is hence explicitly enforced. The method is trained on a large database of 3078 cases from UK Biobank. It is then tested on 756 different cases from UK Biobank and three other state-of-the-art cohorts (ACDC with 100 cases, Sunnybrook with 30 cases, RVSC with 16 cases). Results comparable or even better than the state-of-the-art in terms of distance measures are achieved. They also emphasize the assets of our method, namely enhanced spatial consistency (currently neither considered nor achieved by the state-of-the-art), and the generalization ability to unseen cases even from other databases.

It has been shown recently that convolutional generative adversarial networks (GANs) are able to capture the temporal-pitch patterns in music using the piano-roll representation, which represents music by binary-valued time-pitch matrices. However, existing models can only generate real-valued piano-rolls and require further post-processing (e.g. hard thresholding, Bernoulli sampling) at test time to obtain the final binary-valued results. In this work, we first investigate how the real-valued predictions generated by the generator may lead to difficulties in training the discriminator. To overcome the binarization issue, we propose to append to the generator an additional refiner network, which uses binary neurons at the output layer. The whole network can be trained in a two-stage training setting: the generator and the discriminator are pretrained in the first stage; the refiner network is then trained along with the discriminator in the second stage to refine the real-valued piano-rolls generated by the pretrained generator to binary-valued ones. The proposed model is able to directly generate binary-valued piano-rolls at test time. Experimental results show improvements to the existing models in most of the evaluation metrics. All source code, training data and audio samples can be found at https://salu133445.github.io/bmusegan/ .

Automatic generation of natural language from images has attracted extensive attention. In this paper, we take one step further to investigate generation of poetic language (with multiple lines) to an image for automatic poetry creation. This task involves multiple challenges, including discovering poetic clues from the image (e.g., hope from green), and generating poems to satisfy both relevance to the image and poeticness in language level. To solve the above challenges, we formulate the task of poem generation into two correlated sub-tasks by multi-adversarial training via policy gradient, through which the cross-modal relevance and poetic language style can be ensured. To extract poetic clues from images, we propose to learn a deep coupled visual-poetic embedding, in which the poetic representation from objects, sentiments and scenes in an image can be jointly learned. Two discriminative networks are further introduced to guide the poem generation, including a multi-modal discriminator and a poem-style discriminator. To facilitate the research, we have collected two poem datasets by human annotators with two distinct properties: 1) the first human annotated image-to-poem pair dataset (with 8,292 pairs in total), and 2) to-date the largest public English poem corpus dataset (with 92,265 different poems in total). Extensive experiments are conducted with 8K images generated with our model, among which 1.5K image are randomly picked for evaluation. Both objective and subjective evaluations show the superior performances against the state-of-art methods for poem generation from images. Turing test carried out with over 500 human subjects, among which 30 evaluators are poetry experts, demonstrates the effectiveness of our approach.

Query auto completion (QAC) systems are a standard part of search engines in industry, helping users formulate their query. Such systems update their suggestions after the user types each character, predicting the user's intent using various signals -- one of the most common being popularity. Recently, deep learning approaches have been proposed for the QAC task, to specifically address the main limitation of previous popularity-based methods: the inability to predict unseen queries. In this work we improve previous methods based on neural language modeling, with the goal of building an end-to-end system. We particularly focus on using real-world data by integrating user information for personalized suggestions when possible. We also make use of time information and study how to increase diversity in the suggestions while studying the impact on scalability. Our empirical results demonstrate a marked improvement on two separate datasets over previous best methods in both accuracy and scalability, making a step towards neural query auto-completion in production search engines.

Traditional cross-modal retrieval assumes explicit association of concepts across modalities, where there is no ambiguity in how the concepts are linked to each other, e.g., when we do the image search with a query "dogs", we expect to see dog images. In this paper, we consider a different setting for cross-modal retrieval where data from different modalities are implicitly linked via concepts that must be inferred by high-level reasoning; we call this setting implicit concept association. To foster future research in this setting, we present a new dataset containing 47K pairs of animated GIFs and sentences crawled from the web, in which the GIFs depict physical or emotional reactions to the scenarios described in the text (called "reaction GIFs"). We report on a user study showing that, despite the presence of implicit concept association, humans are able to identify video-sentence pairs with matching concepts, suggesting the feasibility of our task. Furthermore, we propose a novel visual-semantic embedding network based on multiple instance learning. Unlike traditional approaches, we compute multiple embeddings from each modality, each representing different concepts, and measure their similarity by considering all possible combinations of visual-semantic embeddings in the framework of multiple instance learning. We evaluate our approach on two video-sentence datasets with explicit and implicit concept association and report competitive results compared to existing approaches on cross-modal retrieval.

Most image editing tools aren't terribly bright when you ask them to fix a photo. They'll borrow content from adjacent pixels (such as Adobe's recently demonstrated context-aware AI fill), but they can't determine what should have been there -- and that's no good if you're trying to restore a decades-old photo where you know what's absent. NVIDIA might have a solution. It developed a deep learning system that restores photos by determining what should be present in blank or corrupted spaces. If there's a missing eye in a portrait, for instance, it knows to insert one even if the eye area is largely obscured.

Some call it "strong" AI, others "real" AI, "true" AI or artificial "general" intelligence (AGI)... whatever the term (and important nuances), there are few questions of greater importance than whether we are collectively in the process of developing generalized AI that can truly think like a human -- possibly even at a superhuman intelligence level, with unpredictable, uncontrollable consequences. This has been a recurring theme of science fiction for many decades, but given the dramatic progress of AI over the last few years, the debate has been flaring anew with particular intensity, with an increasingly vocal stream of media and conversations warning us that AGI (of the nefarious kind) is coming, and much sooner than we'd think. Latest example: the new documentary Do you trust this computer?, which streamed last weekend for free courtesy of Elon Musk, and features a number of respected AI experts from both academia and industry. The documentary paints an alarming picture of artificial intelligence, a "new life form" on planet earth that is about to "wrap its tentacles" around us. There is also an accelerating flow of stories pointing to an ever scarier aspects of AI, with reports of alternate reality creation (fake celebrity face generator and deepfakes, with full video generation and speech synthesis being likely in the near future), the ever-so-spooky Boston Dynamics videos (latest one: robots cooperating to open a door) and reports about Google's AI getting "highly aggressive" However, as an investor who spends a lot of time in the "trenches" of AI, I have been experiencing a fair amount of cognitive dissonance on this topic.

Adopting a new machine learning algorithm, defining what kind of machine learning experience you need, and how modern astronomy is using machine learning. Your new machine learning algorithm must be trained. According to this data scientist, you could think of it as a new employee with no common sense. Experience with machine learning could mean creating new algorithms or applying existing ones. Researchers are using deep learning algorithms for a more efficient way to search for gravitational waves.

In the past few years, the field of artificial intelligence has exploded with huge leaps and bounds of progress. Scientists have now found a way to take an intelligent AI and use it to analyze images of distant galaxies – revealing key information about galactic evolution. When we learn more about the distant galaxies around us, we learn more about our own Galaxy and how and why it works the way it does. By taking a deep learning algorithm and applying it to the study of other galaxies, we may finally have the tools we need to truly understand galactic evolution. The main way in which this new ability to study galactic evolution is useful is due to the fact that the majority of the galaxies around us formed over billions of years, with our telescopes only capable of giving us a small snapshot of the history of the systems.

" Wait, what?? That's a horrible idea! Were the exact words of our leading NLP researcher when I first talked to her about this concept. Maybe she's right, but it's also definitely a very interesting concept which is getting more attention in the Javascript community lately. During the past year our team is building Bit which makes it simpler to build software using components. As part of our work, we develop ML and NLP algorithms to better understand how code is written, organized and used.