Welcome to my first blog on topics in artificial intelligence! Here I will introduce the topic of edge computing, with context in deep learning applications. This blog is largely adapted from a survey paper written by Xiaofei Wang et al.: Convergence of Edge Computing and Deep Learning: A Comprehensive Survey. If you're interested in learning more about any topic covered here, there are plenty of examples, figures, and explanations in the full 35 page survery: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp & arnumber 8976180 Now, before we begin, I'd like to take a moment and motivate why edge computing and deep learning can be very powerful when combined: Deep learning is becoming an increasingly-capable practice in machine learning that allows computers to detect objects, recognize speech, translate languages, and make decisions. More problems in machine learning are solved with the advanced techniques that researchers discover by the day.

The creation of poems via neural networks is relatively easy nowadays and the internet is replete with corresponding examples. However, it largely lacks interpretive concepts. What should be done with the results generated in this way? How can we draw scientific conclusions from them? This is all the more difficult to answer as it still remains unclear where to position deep‐learning approaches in the canon of digital‐humanities methods. But it is clear that humanities scholars must reckon with machines being responsible for, or at least involved in, the creation of their objects of study.

A new sensing method has made tracking movement easier and more efficient. A research group from Tohoku University has captured dexterous 3D motion data from a flexible magnetic flux sensor array, using deep learning and a structure-aware temporal bilateral filter. "We can now track complex motions with higher accuracy," said Yoshifumi Kitamura, co-author of the study. Dexterous 3D motion data can be used for multiple purposes: biologists can use the data to record detailed movements of small animals in their living environments, scientists can track the flow of fluids, and researchers can track finger movements and objects being manipulated by users in virtual reality. Currently, optical cameras are the most prominent method of tracking movements.

Blog: Open Graph Benchmark is a collection of realistic, large-scale, and diverse benchmark datasets for machine learning on graphs. Network repository is not only the first interactive repository, but also the largest network repository with thousands of donations in 30 domains (from biological to social network data). Blog: Graph Classification: The mission of Papers With Code is to create a free and open resource with Machine Learning papers, code and evaluation tables. Blog: Graph Challenge Data Sets: Amazon is making the Graph Challenge data sets available to the community free of charge as part of the AWS Public Data Sets program. The data is being presented in several file formats, and there are a variety of ways to access it.

The hard part is diversifying the content. So if we just have the same character in an environment doing everything, it's not going to work, right? So how do you actually create hundreds or thousands of variations of that character model with different behavior and things like that? That's been really the core focus of how we're thinking about our technology. You're listening to Gradient Dissent, a show where we learn about making machine learning models work in the real world. Daeil Kim is the co-founder and CEO of AI.Reverie. A startup that specializes in creating high quality synthetic training data for computer vision algorithms. Before that he was a senior data scientist at the New York Times. And before that he got his PhD in computer science from Brown university, focusing on machine learning and Bayesian statistics. He's going to talk about tools that will advance machine learning progress, and he's going to talk about synthetic data. I'm super excited for this. I was looking at your LinkedIn and you have a little bit of an unusual path, right? You did a liberal arts undergrad. Can you say a little bit about... I feel like I come across people quite a lot that want to make career transitions into machine learning and related fields. What was that for you? What prompted you to do it?

Recently, researchers from the Western Kentucky University proposed a multi-modal deep learning framework that has the capability to classify genres of video games based on the cover and textual description. The researchers claimed that this research is the first-ever attempt on automatic genre classification using a deep learning approach. Videos games have been one of the most widespread, profitable, and prominent forms of entertainment around the globe. Also, genre and its classification systems play a significant role in the development of video games. According to the researchers, video game covers and textual descriptions are usually the very first impression to its consumers, and they often convey important information about the video games.

If I wanted to learn deep learning with Python again, I would probably start with PyTorch, an open-source library developed by Facebook's AI Research Lab that is powerful, easy to learn, and very versatile. When it comes to training material, however, PyTorch lags behind TensorFlow, Google's flagship deep learning library. There are fewer books on PyTorch than TensorFlow, and even fewer online courses. Among them is Deep Learning with PyTorch by Eli Stevens, Luca Antiga, and Thomas Viehmann, three engineers who have contributed to the project and have extensive experience developing deep learning solutions. Deep Learning with PyTorch is split across two main sections, first teaching the basics of deep learning and then delving into an advanced, real-world application of medical imaging analysis.

In the winter of 2011, Daniel Yamins, a postdoctoral researcher in computational neuroscience at the Massachusetts Institute of Technology, would at times toil past midnight on his machine vision project. He was painstakingly designing a system that could recognize objects in pictures, regardless of variations in size, position and other properties -- something that humans do with ease. The system was a deep neural network, a type of computational device inspired by the neurological wiring of living brains. "I remember very distinctly the time when we found a neural network that actually solved the task," he said. It was 2 a.m., a tad too early to wake up his adviser, James DiCarlo, or other colleagues, so an excited Yamins took a walk in the cold Cambridge air. "I was really pumped," he said. It would have counted as a noteworthy accomplishment in artificial intelligence alone, one of many that would make neural networks the darlings of AI technology over the next few years.

I prepared this tutorial because it is somehow very difficult to find a blog post with actual working BERT code from the beginning till the end. They are always full of bugs. So, I have dug into several articles, put together their codes, edited them, and finally have a working BERT model. So, just by running the code in this tutorial, you can actually create a BERT model and fine-tune it for sentiment analysis. Natural language processing (NLP) is one of the most cumbersome areas of artificial intelligence when it comes to data preprocessing.

You may remember our previous blog post showing that it is possible to do state-of-the-art deep learning with learning rate that increases exponentially during training. It was meant to be a dramatic illustration that what we learned in optimization classes and books isn't always a good fit for modern deep learning, specifically, normalized nets, which is our term for nets that use any one of popular normalization schemes,e.g. Today's post (based upon our paper with Kaifeng Lyu at NeurIPS20) identifies other surprising incompatibilities between normalized nets and traditional analyses. We hope this will change the way you teach and think about deep learning! Before diving into the results, we recall that normalized nets are typically trained with weight decay (aka $\ell_2$ regularization).