Collaborating Authors


We don't need to worry about Overfitting anymore


Motivated by prior work connecting the geometry of the loss landscape and generalization, we introduce a novel, effective procedure for instead simulta- neously minimizing loss value and loss sharpness. In particular, our procedure, Sharpness-Aware Minimization (SAM), seeks parameters that lie in neighbor- hoods having uniformly low loss; this formulation results in a min-max optimiza- tion problem on which gradient descent can be performed efficiently. We present empirical results showing that SAM improves model generalization across a variety of benchmark datasets[1] In Deep Learning we use optimization algorithms such as SGD/Adam to achieve convergence in our model, which leads to finding the global minima, i.e a point where the loss of the training dataset is low. But several kinds of research such as Zhang et al have shown, many networks can easily memorize the training data and have the capacity to readily overfit, To prevent this problem and add more generalization, Researchers at Google have published a new paper called Sharpness Awareness Minimization which provides State of the Art results on CIFAR10 and other datasets. In this article, we will look at why SAM can achieve better generalization and how we can implement SAM in Pytorch.

AIhub monthly digest: February 2021


Welcome to the second of our monthly digests, designed to keep you up-to-date with the happenings in the AI world. You can catch up with any AIhub stories you may have missed, get the low-down on recent conferences, and generally immerse yourself in all things AI. You may be aware that we are running a focus series on the UN sustainable development goals (SDG). Each month we tackle a different SDG and cover some of the AI research linked to that particular goal. In February it was the turn of climate action.

Potential early diagnostic biomarkers of sepsis


Objective: The goal of this article was to identify potential biomarkers for early diagnosis of sepsis in order to improve their survival. Methods: We analyzed differential gene expression between adult sepsis patients and controls in the GSE54514 dataset. Coexpression analysis was used to cluster coexpression modules, and enrichment analysis was performed on module genes. We also analyzed differential gene expression between neonatal sepsis patients and controls in the GSE25504 dataset, and we identified the subset of differentially expressed genes (DEGs) common to neonates and adults. All samples in the GSE54514 dataset were randomly divided into training and validation sets, and diagnostic signatures were constructed using least absolute shrink and selection operator (LASSO) regression.

Deep learning isn't hard anymore


This had the effect of bottlenecking deep learning, limiting it to the few projects that met those conditions. Over the last couple years, however, things have changed. The driver behind this growth is transfer learning. Transfer learning, broadly, is the idea that the knowledge accumulated in a model trained for a specific task--say, identifying flowers in a photo--can be transferred to another model to assist in making predictions for a different, related task--like identifying melanomas on someone's skin. Note: If you want a more technical dive into transfer learning, Sebastian Ruder has written a fantastic primer.

The Future Is AI: Catalysing Change In Your Business


Artificial intelligence – AI – was a mere computational theory back in the 1950s when Alan Turing designed the first Turing Test to measure a machine's intelligence. Today, AI inhabits consumer electronics in the form of Siri, Cortana, Alexa and Google Assistant – it lives behind our internet browsers, within the relative confines of wireless networks and circuit boards. We interact with AI all the time – Google's auto-suggest function, customer service bots and YouTube's search algorithm are all examples of AI. In just half a century, AI's role in society has become firmly established. Developments in software programming and IT have facilitated important innovations in AI.

Analyzing open-source ML pipeline models in real time using Amazon SageMaker Debugger


Open-source workflow managers are popular because they make it easy to orchestrate machine learning (ML) jobs for productions. Taking models into productions following a GitOps pattern is best managed by a container-friendly workflow manager, also known as MLOps. Kubeflow Pipelines (KFP) is one of the Kubernetes-based workflow managers used today. However, it doesn't provide all the functionality you need for a best-in-class data science and ML engineer experience. A common issue when developing ML models is having access to the tensor-level metadata of how the job is performing.

Using AI to classify a book


We are going to work on a specific sub-task of NLP called text classification, this is the process of recognizing a pattern in a text and assign it a label. Examples that are used in your day to day life without you even noticing it include spam detection (in your mailbox), sentiment analysis (when you review a product or leave a comment) and tagging customer queries (when you fill in a contact form on a website). What we will try to do is to classify science-fiction books into different subgenres (dystopia, cyberpunk, space opera, …) based on their plot. In the end, we want a model that is able to take a book plot as an input and output the subgenres detected in the text and the confidence of the model that a subgenre is detected. The demonstrator can take up to 1 minute to open because I use a free version of Heroku to host my app, thus it goes to sleep when nobody uses it and it's better for the planet! This kind of algorithms could help an online market place to classify the books they receive to make more performant recommendations or a librarian to organize originally the books by subgenres instead of alphabetically, to create an experience in the library. Data is one of the most important (if not the most important) thing in data science.

3 Mathematical Laws Data Scientists Need To Know - KDnuggets


While a Data Scientist works with data as their main activity, it doesn't mean that mathematical knowledge is something we do not need. Data scientists need to learn and understand the mathematical theory behind machine learning to efficiently solving business problems. The mathematics behind machine learning is not just a random notation thrown here and there, but it consists of many theories and thoughts. This thought creates a lot of mathematical laws that contribute to the machine learning we can use right now. Although you could use the mathematics in any way you want to solve the problem, mathematical laws are not limited to machine learning after all.

Career 101: How to Become a Data Scientist with Non-technical Background


The global market revenues from data science activities are set to grow in leaps and bounds in the future. And hence, it is no wonder that the demand for data scientists in various industrial roles will rise in proportion to market growth. But the main question is how to get started for a career in data science? While there are specialized technical courses that can be pursued if one has a technical background, things may not be the same for someone with a non-technical (non-engineering) background. At the same time, given the gap between existing skills and required skills, it will be sometime before a non-techie finds a perfect fit in the data science market. Nevertheless, interested individuals can still succeed professionally with or without a technical background.

How Do Cat Speech Translation Apps Work?


You've probably seen apps that claim to translate what your cat is saying. But can they really translate your cat's meow into English? The short answer is yes, sort of. It's difficult because of how unique each cat's "language" is, but they can get pretty close with modern technology. Cat translation apps like MeowTalk use a form of speech recognition that emphasizes machine learning.