This paper describes a system submitted by team BigGreen to LCP 2021 for predicting the lexical complexity of English words in a given context. We assemble a feature engineering-based model with a deep neural network model founded on BERT. While BERT itself performs competitively, our feature engineering-based model helps in extreme cases, eg. separating instances of easy and neutral difficulty. Our handcrafted features comprise a breadth of lexical, semantic, syntactic, and novel phonological measures. Visualizations of BERT attention maps offer insight into potential features that Transformers models may learn when fine-tuned for lexical complexity prediction. Our ensembled predictions score reasonably well for the single word subtask, and we demonstrate how they can be harnessed to perform well on the multi word expression subtask too.

The growing quantity and complexity of data pose challenges for humans to consume information and respond in a timely manner. For businesses in domains with rapidly changing rules and regulations, failure to identify changes can be costly. In contrast to expert analysis or the development of domain-specific ontology and taxonomies, we use a task-based approach for fulfilling specific information needs within a new domain. Specifically, we propose to extract task-based information from incoming instance data. A pipeline constructed of state of the art NLP technologies, including a bi-LSTM-CRF model for entity extraction, attention-based deep Semantic Role Labeling, and an automated verb-based relationship extractor, is used to automatically extract an instance level semantic structure. Each instance is then combined with a larger, domain-specific knowledge graph to produce new and timely insights. Preliminary results, validated manually, show the methodology to be effective for extracting specific information to complete end use-cases.

We propose TalkNet, a non-autoregressive convolutional neural model for speech synthesis with explicit pitch and duration prediction. The model consists of three feed-forward convolutional networks. The first network predicts grapheme durations. An input text is expanded by repeating each symbol according to the predicted duration. The second network predicts pitch value for every mel frame. The third network generates a mel-spectrogram from the expanded text conditioned on predicted pitch. All networks are based on 1D depth-wise separable convolutional architecture. The explicit duration prediction eliminates word skipping and repeating. The quality of the generated speech nearly matches the best auto-regressive models - TalkNet trained on the LJSpeech dataset got MOS4.08. The model has only 13.2M parameters, almost 2x less than the present state-of-the-art text-to-speech models. The non-autoregressive architecture allows for fast training and inference - 422x times faster than real-time. The small model size and fast inference make the TalkNet an attractive candidate for embedded speech synthesis.

Obtaining labeled data for machine learning tasks can be prohibitively expensive. Active learning mitigates this issue by exploring the unlabeled data space and prioritizing the selection of data that can best improve the model performance. A common approach to active learning is to pick a small sample of data for which the model is most uncertain. In this paper, we explore the efficacy of Bayesian neural networks for active learning, which naturally models uncertainty by learning distribution over the weights of neural networks. By performing a comprehensive set of experiments, we show that Bayesian neural networks are more efficient than ensemble based techniques in capturing uncertainty. Our findings also reveal some key drawbacks of the ensemble techniques, which was recently shown to be more effective than Monte Carlo dropouts.

Learn to create Deep Learning Algorithms in Python from two Machine Learning & Data Science experts. Welcome back to the course on deep learning. Today we're talking about how do neural networks work. Now we've laid a lot of groundwork we've talked about how neural networks are structured, what elements they consist of, and even their functionality. Today we're going to look at a real example of how a neural network can be applied and we're actually gonna work step-by-step through the process of its application so we know what's going on.

In this project, we'll build a model for Anomaly Detection in Time Series data using Deep Learning in Keras with Python code.

Microsoft's recent shopping spree reached a new climax this week with the announcement of its $19.7 billion acquisition of Nuance, a company that provides speech recognition and conversational AI services. Nuance is best known for its deep learning voice transcription service, which is very popular in the health care sector. The two companies had already been working together closely before the acquisition. Nuance had built several of its products on top of Microsoft's Azure cloud. And Microsoft had been using Nuance's Dragon service in its Cloud for Healthcare solution, which launched last year in the midst of the pandemic.

The Amazon ML interview, called the Machine Learning Engineer Interview, focuses heavily on e-commerce ML tools, cloud computing, and AI recommendation systems. Amazon ML engineers are expected to build ML systems and use Deep Learning models. Research scientists have higher levels of education and work to improve ASR, NLU, and TTS features. The technical portion of the ML interview focuses on ML models, bias-variance tradeoff, and overfitting. The Facebook ML Interview consists of generic algorithm questions, ML design, and system design.

The metaverse will be enabled, populated by and supported with artificial intelligence (AI). It will drive all seven technology layers of the metaverse: powering spatial computing, providing scaffolding to creators, and supplying new and sophisticated forms of storytelling. This article will give you a taste of some of these markets, and where we'll see it soonest. Few people realize how quickly AI is advancing. Let's take a look at the exponential growth of Deep Learning Transformers, a type of neural network that allows machines to work with natural language: The original Generative Pre-trained Transformer (GPT) worked with 110 million parameters; the newest Google Brain transformer will go over 1 trillion parameters.

Large pretrained generative models like GPT-3 often suffer from hallucinating non-existent or incorrect content, which undermines their potential merits in real applications. Existing work usually attempts to detect these hallucinations based on a corresponding oracle reference at a sentence or document level. However ground-truth references may not be readily available for many free-form text generation applications, and sentence- or document-level detection may fail to provide the fine-grained signals that would prevent fallacious content in real time. As a first step to addressing these issues, we propose a novel token-level, reference-free hallucination detection task and an associated annotated dataset named HaDes (HAllucination DEtection dataSet). To create this dataset, we first perturb a large number of text segments extracted from English language Wikipedia, and then verify these with crowd-sourced annotations. To mitigate label imbalance during annotation, we utilize an iterative model-in-loop strategy. We conduct comprehensive data analyses and create multiple baseline models.