Deep Learning
A Light-weight contextual spelling correction model for customizing transducer-based speech recognition systems
Wang, Xiaoqiang, Liu, Yanqing, Zhao, Sheng, Li, Jinyu
It's challenging to customize transducer-based automatic In this work, we propose a novel contextual biasing method speech recognition (ASR) system with context information which leverages contextual information by adding a contextual which is dynamic and unavailable during model training. In spelling correction (CSC) model on top of the transducer this work, we introduce a light-weight contextual spelling correction model. To consider contextual information during correction, model to correct context-related recognition errors in a context encoder which encodes context phrases into hidden transducer-based ASR systems. We incorporate the context information embeddings is added to the spelling correction model [16, 17], into the spelling correction model with a shared context the decoder of the correction model then attends to the context encoder and use a filtering algorithm to handle large-size encoder and text encoder by attention mechanism [18].
One-shot Transfer Learning for Population Mapping
Shao, Erzhuo, Feng, Jie, Wang, Yingheng, Xia, Tong, Li, Yong
Fine-grained population distribution data is of great importance for many applications, e.g., urban planning, traffic scheduling, epidemic modeling, and risk control. However, due to the limitations of data collection, including infrastructure density, user privacy, and business security, such fine-grained data is hard to collect and usually, only coarse-grained data is available. Thus, obtaining fine-grained population distribution from coarse-grained distribution becomes an important problem. To tackle this problem, existing methods mainly rely on sufficient fine-grained ground truth for training, which is not often available for the majority of cities. That limits the applications of these methods and brings the necessity to transfer knowledge between data-sufficient source cities to data-scarce target cities. In knowledge transfer scenario, we employ single reference fine-grained ground truth in target city, which is easy to obtain via remote sensing or questionnaire, as the ground truth to inform the large-scale urban structure and support the knowledge transfer in target city. By this approach, we transform the fine-grained population mapping problem into a one-shot transfer learning problem. In this paper, we propose a novel one-shot transfer learning framework PSRNet to transfer spatial-temporal knowledge across cities from the view of network structure, the view of data, and the view of optimization. Experiments on real-life datasets of 4 cities demonstrate that PSRNet has significant advantages over 8 state-of-the-art baselines by reducing RMSE and MAE by more than 25%. Our code and datasets are released in Github (https://github.com/erzhuoshao/PSRNet-CIKM).
Unifying Nonlocal Blocks for Neural Networks
Zhu, Lei, She, Qi, Li, Duo, Lu, Yanye, Kang, Xuejing, Hu, Jie, Wang, Changhu
The nonlocal-based blocks are designed for capturing long-range spatial-temporal dependencies in computer vision tasks. Although having shown excellent performance, they still lack the mechanism to encode the rich, structured information among elements in an image or video. In this paper, to theoretically analyze the property of these nonlocal-based blocks, we provide a new perspective to interpret them, where we view them as a set of graph filters generated on a fully-connected graph. Specifically, when choosing the Chebyshev graph filter, a unified formulation can be derived for explaining and analyzing the existing nonlocal-based blocks (e.g., nonlocal block, nonlocal stage, double attention block). Furthermore, by concerning the property of spectral, we propose an efficient and robust spectral nonlocal block, which can be more robust and flexible to catch long-range dependencies when inserted into deep neural networks than the existing nonlocal blocks. Experimental results demonstrate the clear-cut improvements and practical applicabilities of our method on image classification, action recognition, semantic segmentation, and person re-identification tasks.
Conditional Variational Capsule Network for Open Set Recognition
Guo, Yunrui, Camporese, Guglielmo, Yang, Wenjing, Sperduti, Alessandro, Ballan, Lamberto
In open set recognition, a classifier has to detect unknown classes that are not known at training time. In order to recognize new categories, the classifier has to project the input samples of known classes in very compact and separated regions of the features space for discriminating samples of unknown classes. Recently proposed Capsule Networks have shown to outperform alternatives in many fields, particularly in image recognition, however they have not been fully applied yet to open-set recognition. In capsule networks, scalar neurons are replaced by capsule vectors or matrices, whose entries represent different properties of objects. In our proposal, during training, capsules features of the same known class are encouraged to match a pre-defined gaussian, one for each class. To this end, we use the variational autoencoder framework, with a set of gaussian priors as the approximation for the posterior distribution. In this way, we are able to control the compactness of the features of the same class around the center of the gaussians, thus controlling the ability of the classifier in detecting samples from unknown classes. We conducted several experiments and ablation of our model, obtaining state of the art results on different datasets in the open set recognition and unknown detection tasks.
What to expect from OpenAI's Codex API
The Transform Technology Summits start October 13th with Low-Code/No Code: Enabling Enterprise Agility. OpenAI will make Codex, its AI programmer technology, available through an application programming interface, the company announced on its blog on Tuesday. In tandem with the announcement, OpenAI CTO Greg Brockman, Chief Scientist Ilya Sutskever, and co-founder Wojciech Zaremba gave an online presentation of the capabilities of the deep learning model. The Codex demo puts the advantages of large language models to full display, showing an impressive capacity to resolve references and write code for a variety of APIs and micro-tasks that can be frustratingly time-consuming. OpenAI is still testing the waters with Codex.
TensorFlow 2.0 Practical
Artificial Intelligence (AI) revolution is here and TensorFlow 2.0 is finally here to make it happen much faster! TensorFlow 2.0 is Google's most powerful, recently released open source platform to build and deploy AI models in practice. AI technology is experiencing exponential growth and is being widely adopted in the Healthcare, defense, banking, gaming, transportation and robotics industries. The purpose of this course is to provide students with practical knowledge of building, training, testing and deploying Artificial Neural Networks and Deep Learning models using TensorFlow 2.0 and Google Colab. The course provides students with practical hands-on experience in training Artificial Neural Networks and Convolutional Neural Networks using real-world dataset using TensorFlow 2.0 and Google Colab.
In-Depth Guide to Web Scraping for Machine Learning in 2021
As of January 2021, 4.7 billion people around the world have been recorded to use the internet, creating 1.7MB of data every second. Crawling this exponentially growing volume of data could provide many opportunities for breakthroughs in data science. Data scientists can leverage crawled data to perform many tasks like real-time analytics, training predictive machine learning models, and improving natural language processing capabilities. Web crawling software, such as Bright Data's data collector, extracts real-time public data from online platforms and deliver it to businesses on autopilot in different formats. This software is especially useful when collecting data from websites that protect themselves against scraping.
Top 15 YouTube Channels To Follow For Deep Learning Enthusiasts
Deep Learning is a subset of machine learning that encompasses neural networks that can learn from raw or unstructured data, much like humans. It's used for speech recognition, machine translation, computer vision and natural language processing. Deep Learning is at the centre of exciting innovation possibilities like Self Driven Cars, Image recognition, virtual assistants, speech recognition, machine translation, computer vision and natural language processing. Deep learning models are transforming businesses by providing learning techniques and real-world solutions based on large data sets. It is good to have skills for today's machine learning job market.
Introduction to Natural Language Processing for Machine Learning
There is a lot of text present around us. We see it in books, articles, comments, and newspapers. It would be really wise to use this text and convert it into a form that could be easily understood by machine learning and deep learning algorithms. As a result, they would take the processed text and give predictions for different use cases. Natural language processing (NLP) refers to converting natural text into a form that could be used for machine learning purposes.