"The field of Machine Learning seeks to answer these questions: How can we build computer systems that automatically improve with experience, and what are the fundamental laws that govern all learning processes?"
– from The Discipline of Machine Learning by Tom Mitchell. CMU-ML-06-108, 2006.
Do you understand how your machine learning model works? Despite the ever-increasing usage of machine learning (ML) and deep learning (DL) techniques, the majority of companies say they can't explain the decisions of their ML algorithms . This is, at least in part, due to the increasing complexity of both the data and models used. It's not easy to find a nice, stable aggregation over 100 decision trees in a random forest to say which features were most important or how the model came to the conclusion it did. This problem grows even more complex in application domains such as computer vision (CV) or natural language processing (NLP), where we no longer have the same high-level, understandable features to help us understand the model's failures.
In a recent post on BERT, we discussed BERT transformers and how they work on a basic level. The article covers BERT architecture, training data, and training tasks. However, we don't really understand something before we implement it ourselves. So in this post, we will implement a Question Answering Neural Network using BERT and a Hugging Face Library. In this task, we are given a question and a paragraph in which the answer lies to our BERT Architecture and the objective is to determine the start and end span for the answer in the paragraph.
AI systems are becoming increasingly popular and central in many industries. They decide who might get a loan from the bank, whether an individual should be convicted, and we may even entrust them with our lives when using systems such as autonomous vehicles in the near future. Thus, there is a growing need for mechanisms to harness and control these systems so that we may ensure that they behave as desired. One important issue that has been gaining popularity in the last few years is fairness. While usually ML models are evaluated based on metrics such as accuracy, the idea of fairness is that we must ensure that our models are unbiased with regard to attributes such as gender, race and other selected attributes.
If you're a deep learning enthusiast you're probably already familiar with some of the basic mathematical primitives that have been driving the impressive capabilities of what we call deep neural networks. Although we like to think of a basic artificial neural network as some nodes with some weighted connections, it's more efficient computationally to think of neural networks as matrix multiplication all the way down. We might draw a cartoon of an artificial neural network like the figure below, with information traveling in from left to right from inputs to outputs (ignoring recurrent networks for now). This type of neural network is a feed-forward multilayer perceptron (MLP). If we want a computer to compute the forward pass for this model, it's going to use a string of matrix multiplies and some sort of non-linearity (here represented by the Greek letter sigma) in the hidden layer: MLPs are well-suited for data that can be naturally shaped as 1D vectors.
The authors concluded that a 178MB AlexNet model can have up to 36.9MB of malware embedded into its structure without being detected using a technique called steganography. Neural networks could be the next frontier for malware campaigns as they become more widely used, according to a new study. According to the study, which was posted to the arXiv preprint server on Monday, malware can be embedded directly into the artificial neurons that make up machine learning models in a way that keeps them from being detected. The neural network would even be able to continue performing its set tasks normally. "As neural networks become more widely used, this method will be universal in delivering malware in the future," the authors, from the University of the Chinese Academy of Sciences, write.
The human mediator complex has long been one of the most challenging multi-protein systems for structural biologists to understand.Credit: Yuan He The human genome holds the instructions for more than 20,000 proteins. But only about one-third of those have had their 3D structures determined experimentally. And in many cases, those structures are only partially known. Now, a transformative artificial intelligence (AI) tool called AlphaFold, which has been developed by Google's sister company DeepMind in London, has predicted the structure of nearly the entire human proteome (the full complement of proteins expressed by an organism). In addition, the tool has predicted almost complete proteomes for various other organisms, ranging from mice and maize (corn) to the malaria parasite (see'Folding options').
Traditionally, African lenders use credit bureau scores, assessing, for instance, if a customer has a history of missed credit card payments. When no history exists, they evaluate social demographics: is the customer female – in which case they are likelier to repay – do they work in a stable job market and can they prove a regular income? But this can put those who are unbanked or informally employed at a disadvantage. Michele Tucci, chief product officer at fintech company Credolab said: "African lenders lack data to make good credit decisions and social demographic data can bring you only so far." He estimates that African lenders cannot obtain credit bureau scores for 70% of customers and simply reject them.
This article will go into more details of node embeddings. If you lack intuition and understanding of node embeddings, check out this previous article that covered the intuition of node embeddings. But if you are ready. In the level-1 explanation of node embeddings I motivated why we need embeddings such that we have a vector form of graph data. Embeddings should capture the graph topology, relationships between nodes and further information.
Marcelo Sousa, a biochemist at the University of Colorado Boulder, had spent ten years trying to crack a particularly tricky puzzle. Sousa and his team had collected reams of experimental data on a single bacterial protein linked to antibiotic resistance. Working out its structure, they hoped, would help to find inhibitors that could stop that resistance from building. But, year after year, the puzzle remained unsolved. Within 15 minutes, DeepMind's machine learning system had solved the structure.