Deep Learning
AI Can Almost Write Like a Human--and More Advances Are Coming
Last month, software developer Kevin Lacker tested GPT-3, the latest version of an artificial-intelligence language system developed by San Francisco-based software company OpenAI LP. The system isn't yet public, but it set off a firestorm in tech circles after OpenAI gave select researchers and developers access so they could provide feedback. They observed its uncanny and unprecedented ability to answer trivia questions, generate long passages of coherent text, design simple software applications and offer plausible recipes for breakfast burritos. Trained on roughly 300 billion words from across the internet, GPT-3 predicts what is most likely to follow a prompt from a human. But ask it to reason, and it struggles.
GitHub unveils AI coding assistant for Visual Studio Code
GitHub has launched a preview of GitHub Copilot, an AI-based coding assitant for Visual Studio Code that suggests lines of code or functions as you type. Built in collaboration with OpenAI, GitHub Copilot draws context from the developer's code, suggesting lines or entire functions while helping to find alternative ways to solve problems, write tests, and explore new APIs without the need to search for answers on the Internet. Introduced June 29, GitHub Copilot adapts to how the user writes code, helping complete work faster. Trained on billions of lines of public code, the tool is powered by OpenAI Codex, an AI system that is more capable than the GPT-3 (Generative Pretrained Transformer) language model in code generation, GitHub said. GitHub Copilot can quickly produce boilerplate code and repetitive patterns, with developers able to feed examples to Copilot and have the tool generate the rest.
OpenAI's gigantic GPT-3 hints at the limits of language models for AI
A little over a year ago, OpenAI, an artificial intelligence company based in San Francisco, stunned the world by showing a dramatic leap in what appeared to be the power of computers to form natural-language sentences, and even to solve questions, such as completing a sentence, and formulating long passages of text people found fairly human. The latest work from that team shows how OpenAI's thinking has matured in some respects. GPT-3, as the newest creation is called, emerged last week, with more bells and whistles, created by some of the same authors as the last version, including Alec Radford and Ilya Sutskever, along with several additional collaborators, including scientists from Johns Hopkins University. It is now a truly monster language model, as it's called, gobbling two orders of magnitude more text than its predecessor. But within that bigger-is-better stunt, the OpenAI team seem to be approaching some deeper truths, much the way Dr. David Bowman approached the limits of the known at the end of the movie 2001.
A New AI Study May Explain Why Deep Learning Works
The resurgence of artificial intelligence (AI) is largely due to advances in pattern-recognition due to deep learning, a form of machine learning that does not require explicit hard-coding. The architecture of deep neural networks is somewhat inspired by the biological brain and neuroscience. Like the biological brain, the inner workings of exactly why deep networks work are largely unexplained, and there is no single unifying theory. Recently researchers at the Massachusetts Institute of Technology (MIT) revealed new insights about how deep learning networks work to help further demystify the black box of AI machine learning. The MIT research trio of Tomaso Poggio, Andrzej Banburski, and Quianli Liao at the Center for Brains, Minds, and Machines developed a new theory as to why deep networks work and published their study published on June 9, 2020 in PNAS (Proceedings of the National Academy of Sciences of the United States of America).
The Machine Learning Research Championed by the Biggest AI Labs in the World - KDnuggets
I recently started a new newsletter focus on AI education and already has over 50,000 subscribers. TheSequence is a no-BS( meaning no hype, no news etc) AI-focused newsletter that takes 5 minutes to read. The goal is to keep you up to date with machine learning projects, research papers and concepts. Recently, one of my students asked me a question as of whether DeepMind was solely working in reinforcement learning applications. The answer is obviously no but the question is still valid as it rooted in the fact that most of DeepMind's highly publicized work such as AlphaGo, MuZero or AlphaFold are based in reinforcement learning.
Using AntiPatterns to avoid MLOps Mistakes
Different values of hyper-parameters often prove to be significant drivers of model performance and are expensive to tune and mostly task specific. Hyper-parameters play such a crucial role in modeling architectures that entire research efforts are devoted to developing efficient hyper-parameter search strategies (Bergstra et al., 2013; Nguyen et al., 2019; Henderson et al., 2018; Van Rijn and Hutter, 2018; Probst et al., 2019). The set of hyper-parameters differs for different learning algorithms. For instance, even a simple classification model like the decision tree classifier, has hyper-parameters like the maximum depth of the tree, the minimum number of samples to split an internal node and the criterion to use for estimating either the impurity at a node (gini) or the information gain (entropy) at each node. Ensemble models like random forest classifiers and gradient boosting machines also have additional parameters governing the number of estimators (trees) to include in the model.
A multi-stage machine learning model on diagnosis of esophageal manometry
High-resolution manometry (HRM) is the primary procedure used to diagnose esophageal motility disorders. Its interpretation and classification includes an initial evaluation of swallow-level outcomes and then derivation of a study-level diagnosis based on Chicago Classification (CC), using a tree-like algorithm. This diagnostic approach on motility disordered using HRM was mirrored using a multi-stage modeling framework developed using a combination of various machine learning approaches. Specifically, the framework includes deep-learning models at the swallow-level stage and feature-based machine learning models at the study-level stage. In the swallow-level stage, three models based on convolutional neural networks (CNNs) were developed to predict swallow type, swallow pressurization, and integrated relaxation pressure (IRP).
Introduction to Generative Adversarial Networks using Pytorch: BollyGAN
"Generative Adversarial Networks is the most interesting idea in the last 10 years in Machine Learning." Ever heard of "Fake it till you make it!!!"? Generative Deep Learning models do exactly that but with the help of mathematics, statistics and data. It works by learning the latent variables which are the variables responsible for the generation of the input data and then using this latent space to generate synthetic data. Basically, Generative modeling is an unsupervised learning task in Deep learning that involves for a given input, model learning the probability distribution of the variables making up this input and generate something new -- either resembling the input or a new synthetic output. Generative Deep Learning models have been employed in a wide range of applications ranging from language synthesis with LSTM to Google's DeepDream algorithm to executing neural style transfers to generating DeepFakes and new dataset curation.