At root, ESZSL is a linear model. Given an input matrix X with shape (number of rows, number of features) and a weight matrix of shape (number of features, number of classes), the linear combination output would be of the shape (number of rows, number of classes). The goal of ESZSL is to find the value of the weight matrix W.

The year 2020 saw many exciting developments in machine learning. As the year 2020 comes to an end, here is a roundup of these innovations in various machine learning domains such as reinforcement learning, Natural Language Processing, ML frameworks such as Pytorch and TensorFlow, and more. Arm-based Graviton processors went mainstream in 2020, which utilize 30 billion transistors with 64-bit Arm cores built by Israeli-based engineering company Annapurna Labs. AWS recently acquired it for powering memory-intensive workloads like real-time big data analytics. It showed a 40% performance improvement emerging as an alternative to x86-based processors for machine learning, shifting the trend from the Intel-dominated cloud market to Arm-based Graviton processors.

This multi-part feature should provide you with a very basic understanding of what AI is, what it can do, and how it works. The guide contains articles on (in order published) neural networks, computer vision, natural language processing, algorithms, artificial general intelligence, and the difference between video game AI and real AI. As legend has it, a reporter once asked Mahatma Ghandi what he thought of Western Civilization. His response was "I think it would be a good idea." The same sentiment could be applied to artificial intelligence if you compare it directly to human intelligence.

Quebec Artificial Intelligence Institute (Mila) and IBM have teamed up to accelerate artificial intelligence (AI) and machine learning research using open-source technology. Mila and IBM have been collaborating since early 2020 on a project that is meant to make a key component of AI, known as hyperparameter optimization, more accessible. The organizations claim that this would improve machine learning model performances and pinpoint within the'black box' of AI where models need work. "A collaboration with…IBM is a great opportunity to accelerate the development of an open-source solution…initiated at Mila." – Yoshua Bengio, Mila The two organizations are looking to integrate the Quebec institute's open-source software, Oríon, with IBM's Watson Machine Learning Accelerator, an AI model training and inference tool that the tech giant offers to businesses. The overall goal, they claim, is to "improve the development, deployment, and ongoing management of complex AI and deep learning models, as well as to make tools more accessible to a larger base of scientists, engineers, and developers through automation."

Keras is a Deep Learning API of TensorFlow 2.0 used for easy and fast experimentation. It is simple to understand, flexible to extend and deploy, and powerful enough to build any neural network. With the increase in the usage of deep learning to solve real-time problems, it has become quite a necessity to lessen the time consumed to build robust machine learning algorithms, i.e., the time taken from designing an algorithm to putting it into practice to generate the desired model has to be minimal. Keras has been designed for this very purpose. It is a high-level deep learning API running on top of TensorFlow, a machine learning/deep learning framework.

A group of researchers from MIT Computer Science & Artificial Intelligence Laboratory (CSAIL) have proposed a simple framework for performing different image reconstruction tasks using the state-of-the-art generative model StyleGAN2. It's common for machine learning researchers to train models in a supervised setting for solving downstream prediction and image reconstruction tasks. For example, in the task of super-resolution, which aims to obtain high-resolution output images from low-resolution versions, classical methods train models on pairs of low-resolution and high-resolution images. However, such end-to-end methods can also require re-training whenever there is a distribution shift in the inputs or relevant latent variables. Distribution shifts can easily occur for example in the input x-ray images collected from a hospital if the hospital's medical scanners are upgraded, or as the patients contributing the images age due to improved healthcare. Given the prohibitively high computation resources required to re-train end-to-end approaches when distribution shifts occur, how else might researchers build ML models that are both easy to train and robust to distribution shifts?

Amazon.com Inc. said this week that after years of research its Alexa voice assistant can now figure out the meaning of requests it has never heard before. The upgrade, which the company calls interactive teaching, could represent a significant advance in the way AI-powered voice assistants interpret and learn from everyday conversation, experts say. Interactive teaching is powered by deep-learning models, and it works by having Alexa ask questions about a task-relevant phrase it is encountering for the first time. For instance, if a user asks Alexa to set the lights to "reading mode" and the device hasn't heard that phrase before, it will ask what it means. If the user says it means to set the lights at 50% brightness, Alexa will remember that for the next time.

Relation Extraction (RE) is to predict the relation type of two entities that are mentioned in a piece of text, e.g., a sentence or a dialogue. When the given text is long, it is challenging to identify indicative words for the relation prediction. Recent advances on RE task are from BERT-based sequence modeling and graph-based modeling of relationships among the tokens in the sequence. In this paper, we propose to construct a latent multi-view graph to capture various possible relationships among tokens. We then refine this graph to select important words for relation prediction. Finally, the representation of the refined graph and the BERT-based sequence representation are concatenated for relation extraction. Specifically, in our proposed GDPNet (Gaussian Dynamic Time Warping Pooling Net), we utilize Gaussian Graph Generator (GGG) to generate edges of the multi-view graph. The graph is then refined by Dynamic Time Warping Pooling (DTWPool). On DialogRE and TACRED, we show that GDPNet achieves the best performance on dialogue-level RE, and comparable performance with the state-of-the-arts on sentence-level RE.

Recurrent Neural Networks (RNNs) have achieved remarkable performance on a range of tasks. A key step to further empowering RNN-based approaches is improving their explainability and interpretability. In this work we present MEME: a model extraction approach capable of approximating RNNs with interpretable models represented by human-understandable concepts and their interactions. We demonstrate how MEME can be applied to two multivariate, continuous data case studies: Room Occupation Prediction, and In-Hospital Mortality Prediction. Using these case-studies, we show how our extracted models can be used to interpret RNNs both locally and globally, by approximating RNN decision-making via interpretable concept interactions.