The development of graph neural networks (GCN) makes it possible to learn structural features from evolving complex networks. Even though a wide range of realistic networks are directed ones, few existing works investigated the properties of directed and temporal networks. In this paper, we address the problem of temporal link prediction in directed networks and propose a deep learning model based on GCN and self-attention mechanism, namely TSAM. The proposed model adopts an autoencoder architecture, which utilizes graph attentional layers to capture the structural feature of neighborhood nodes, as well as a set of graph convolutional layers to capture motif features. A graph recurrent unit layer with self-attention is utilized to learn temporal variations in the snapshot sequence. We run comparative experiments on four realistic networks to validate the effectiveness of TSAM. Experimental results show that TSAM outperforms most benchmarks under two evaluation metrics.

As applications in large organizations evolve, the machine learning (ML) models that power them must adapt the same predictive tasks to newly arising data modalities (e.g., a new video content launch in a social media application requires existing text or image models to extend to video). To solve this problem, organizations typically create ML pipelines from scratch. However, this fails to utilize the domain expertise and data they have cultivated from developing tasks for existing modalities. We demonstrate how organizational resources, in the form of aggregate statistics, knowledge bases, and existing services that operate over related tasks, enable teams to construct a common feature space that connects new and existing data modalities. This allows teams to apply methods for training data curation (e.g., weak supervision and label propagation) and model training (e.g., forms of multi-modal learning) across these different data modalities. We study how this use of organizational resources composes at production scale in over 5 classification tasks at Google, and demonstrate how it reduces the time needed to develop models for new modalities from months to weeks to days.

Captcha is computer generating text images used to distinguish interactions given by humans or machines. Normally, a captcha image consists of a fixed number of characters (e.g. These characters are not only distorted, scaled into multiple different sizes but also can be overlapped and crossed by multiple random lines. Two types of captcha illustrated in Figure 1, are specified by the number of character categories (0..9, A..Z) and the text length (e.g. 5 in green and 6 in black images). In this blog, I present and compare two deep learning models solving this captcha recognition challenge.

Since OpenAI first described its new AI language-generating system called GPT-3 in May, hundreds of media outlets (including MIT Technology Review) have written about the system and its capabilities. Twitter has been abuzz about its power and potential. The New York Times published an op-ed about it. Later this year, OpenAI will begin charging companies for access to GPT-3, hoping that its system can soon power a wide variety of AI products and services. Is GPT-3 an important step toward artificial general intelligence--the kind that would allow a machine to reason broadly in a manner similar to humans without having to train for every specific task it encounters?

TensorBoard is one of the best tools for visualizing the training stats of your deep learning models. I like to have TensorBoard running on my phone so I can check it on the go. It's very simple to set up and can give you a boost of adrenaline as you monitor your training accuracy at the gym. All you need to do is download a package called ngrok which will expose a local port over a secure tunnel to a public internet address. We can use this to tunnel TensorBoard's port 6006 to a phone, computer, or any internet connected device.

Skin conditions affect 1.9 billion people. Because of a shortage of dermatologists, most cases are seen instead by general practitioners with lower diagnostic accuracy. We present a deep learning system (DLS) to provide a differential diagnosis of skin conditions using 16,114 de-identified cases (photographs and clinical data) from a teledermatology practice serving 17 sites. The DLS distinguishes between 26 common skin conditions, representing 80% of cases seen in primary care, while also providing a secondary prediction covering 419 skin conditions. On 963 validation cases, where a rotating panel of three board-certified dermatologists defined the reference standard, the DLS was non-inferior to six other dermatologists and superior to six primary care physicians (PCPs) and six nurse practitioners (NPs) (top-1 accuracy: 0.66 DLS, 0.63 dermatologists, 0.44 PCPs and 0.40 NPs). These results highlight the potential of the DLS to assist general practitioners in diagnosing skin conditions. A deep learning system able to identify the most common skin conditions may help clinicians in making more accurate diagnoses in routine clinical practice

Statsmodels is an open-source statistics-driven module that offers various classes and functions to the many statistical models available for statistical analysis and exploration of data. The module covers a vast number of models ranging from Linear Regression, Discrete Models, Time Series Analysis, Survival Analysis, and many other miscellaneous models.

These are the lecture notes for FAU's YouTube Lecture "Deep Learning". This is a full transcript of the lecture video & matching slides. We hope, you enjoy this as much as the videos. Of course, this transcript was created with deep learning techniques largely automatically and only minor manual modifications were performed. If you spot mistakes, please let us know! Welcome back to deep learning! So today, we want to start talking about ideas that are called self-supervised learning.

In June, OpenAI unveiled the largest language model in the world, a text-generating tool called GPT-3 that can write creative fiction, translate legalese into plain English, and answer obscure trivia questions. It's the latest feat of intelligence achieved by deep learning, a machine learning method patterned after the way neurons in the brain process and store information. But it came at a hefty price: at least $4.6 million and 355 years in computing time, assuming the model was trained on a standard neural network chip, or GPU. The model's colossal size -- 1,000 times larger than a typical language model -- is the main factor in its high cost. "You have to throw a lot more computation at something to get a little improvement in performance," says Neil Thompson, an MIT researcher who has tracked deep learning's unquenchable thirst for computing.

"If I were not a physicist, I would probably be a musician. I often think in music. I live my daydreams in music. I see my life in terms of music." I might not be a physicist like Mr. Einstein, but I wholeheartedly agree with his thoughts on music!