If you are new to computer vision or an experienced AI engineer prototyping a real-world application, Vision AI Development Kit is the tool you need.

Wallner: My name is Emil Wallner, I'm Swedish, and I'm currently studying computer science in Paris. You might have come across some of my open-source projects. I made the one that you mentioned earlier, the Screenshot-to-code, which was the most popular project on GitHub for almost a month. That's where I translate design markups into HTML and CSS. Another project I've done is coloring black and white photos with neural networks and [inaudible 00:00:30] made a short film about this project. Today, we're going to talk about software automation. The first step you need to take to start understanding this problem is to start looking at software development as data. A lot of people now, if you think about the tasks that we're given, say, design markups, or program descriptions, or meeting a client and trying to understand what their problems are, we see them as human problems that only we can relate to and understand, but more and more, we can start to treat these problems as data problems. To understand how this is possible, I'm just going to give you a short overview of the context. We've had traditional software that we're all used to.

The beach offers a wide open playscape where children are fueled by curiosity.

Convolution is at the heart of deep learning. It is a particularly important mathematical operation intrinsically related with the mixing and/or correlation between two functions.

Daniel Konhauser12:21 PM are you leveraging transfer learning in these demos? Daniel Situnayake12:21 PM @Duncan our speech model that can detect "yes" or "no" is 20kb, our vision model that can detect the presence or absence of a person is 250kb. Pete Warden12:21 PM @Tara we're actually working on a book that has a chapter about setting up TF Lite for new devices. TF Lite Micro should be a lot easier to port than regular TF Lite too - http://shop.oreilly.com/product/0636920254508.do Meghna Natraj12:22 PM @Andres Manjarres Unfortunately no. 1) we train regular tensor flow models (and factor in micro controller constraints if possible- small model size, low latency) 2) convert it to a tensorflow lite model 3) run it on a microcontroller that we support. Pete Warden12:22 PM @Tara drop an email to petewarden@google.com and we might be able to send you a draft copy Dick Brooks12:23 PM Question: Do you plan to build a Deep Learning demo also?

Skin conditions are among the most common kind of ailment globally, just behind colds, fatigue, and headaches. In fact, it's estimated that 25% of all treatments provided to patients around the world are for skin conditions and that up to 37% of patients seen in the clinic have at least one skin complaint. The enormous case workload and a global shortage of dermatologists have forced sufferers to seek out general practitioners, who tend to be less accurate than specialists when it comes to identifying conditions. This trend motivated researchers at Google to investigate an AI system capable of spotting the most common dermatological disorders seen in primary care. In a paper ("A Deep Learning System for Differential Diagnosis of Skin Diseases") and accompanying blog post, they report that it achieves accuracy across 26 skin conditions when presented with images and metadata about a patient case, and they claim that it's on par with U.S. board-certified dermatologists.

Mobile agents that can leverage help from humans can potentially accomplish more complex tasks than they could entirely on their own. We develop "Help, Anna!" (HANNA), an interactive photo-realistic simulator in which an agent fulfills object-finding tasks by requesting and interpreting natural language-and-vision assistance. An agent solving tasks in a HANNA environment can leverage simulated human assistants, called ANNA (Automatic Natural Navigation Assistants), which, upon request, provide natural language and visual instructions to direct the agent towards the goals. To address the HANNA problem, we develop a memory-augmented neural agent that hierarchically models multiple levels of decision-making, and an imitation learning algorithm that teaches the agent to avoid repeating past mistakes while simultaneously predicting its own chances of making future progress. Empirically, our approach is able to ask for help more effectively than competitive baselines and, thus, attains higher task success rate on both previously seen and previously unseen environments. We publicly release code and data at https://github.com/khanhptnk/hanna .

We are enveloped by stories of visual interpretations in our everyday lives. The way we narrate a story often comprises of two stages, which are, forming a central mind map of entities and then weaving a story around them. A contributing factor to coherence is not just basing the story on these entities but also, referring to them using appropriate terms to avoid repetition. In this paper, we address these two stages of introducing the right entities at seemingly reasonable junctures and also referring them coherently in the context of visual storytelling. The building blocks of the central mind map, also known as entity skeleton are entity chains including nominal and coreference expressions. This entity skeleton is also represented in different levels of abstractions to compose a generalized frame to weave the story. We build upon an encoder-decoder framework to penalize the model when the decoded story does not adhere to this entity skeleton. We establish a strong baseline for skeleton informed generation and then extend this to have the capability of multitasking by predicting the skeleton in addition to generating the story. Finally, we build upon this model and propose a glocal hierarchical attention model that attends to the skeleton both at the sentence (local) and the story (global) levels. We observe that our proposed models outperform the baseline in terms of automatic evaluation metric, METEOR. We perform various analysis targeted to evaluate the performance of our task of enforcing the entity skeleton such as the number and diversity of the entities generated. We also conduct human evaluation from which it is concluded that the visual stories generated by our model are preferred 82% of the times. In addition, we show that our glocal hierarchical attention model improves coherence by introducing more pronouns as required by the presence of nouns.

In the few-shot scenario, a learner must effectively generalize to unseen classes given a small support set of labeled examples. While a relatively large amount of research has gone into few-shot learning for image classification, little work has been done on few-shot video classification. In this work, we address the task of few-shot video action recognition with a set of two-stream models. We evaluate the performance of a set of convolutional and recurrent neural network video encoder architectures used in conjunction with three popular metric-based few-shot algorithms. We train and evaluate using a few-shot split of the Kinetics 600 dataset. Our experiments confirm the importance of the two-stream setup, and find prototypical networks and pooled long short-term memory network embeddings to give the best performance as few-shot method and video encoder, respectively. For a 5-shot 5-way task, this setup obtains 84.2% accuracy on the test set and 59.4% on a special "challenge" test set, composed of highly confusable classes.