This paper describes the submissions by team HWR to the Dravidian Language Identification (DLI) shared task organized at VarDial 2021 workshop. The DLI training set includes 16,674 YouTube comments written in Roman script containing code-mixed text with English and one of the three South Dravidian languages: Kannada, Malayalam, and Tamil. We submitted results generated using two models, a Naive Bayes classifier with adaptive language models, which has shown to obtain competitive performance in many language and dialect identification tasks, and a transformer-based model which is widely regarded as the state-of-the-art in a number of NLP tasks. Our first submission was sent in the closed submission track using only the training set provided by the shared task organisers, whereas the second submission is considered to be open as it used a pretrained model trained with external data. Our team attained shared second position in the shared task with the submission based on Naive Bayes. Our results reinforce the idea that deep learning methods are not as competitive in language identification related tasks as they are in many other text classification tasks.

This paper proposes a model-based framework to automatically and efficiently design understandable and verifiable behaviors for swarms of robots. The framework is based on the automatic extraction of two distinct models: 1) a neural network model trained to estimate the relationship between the robots' sensor readings and the global performance of the swarm, and 2) a probabilistic state transition model that explicitly models the local state transitions (i.e., transitions in observations from the perspective of a single robot in the swarm) given a policy. The models can be trained from a data set of simulated runs featuring random policies. The first model is used to automatically extract a set of local states that are expected to maximize the global performance. These local states are referred to as desired local states. The second model is used to optimize a stochastic policy so as to increase the probability that the robots in the swarm observe one of the desired local states. Following these steps, the framework proposed in this paper can efficiently lead to effective controllers. This is tested on four case studies, featuring aggregation and foraging tasks. Importantly, thanks to the models, the framework allows us to understand and inspect a swarm's behavior. To this end, we propose verification checks to identify some potential issues that may prevent the swarm from achieving the desired global objective. In addition, we explore how the framework can be used in combination with a "standard" evolutionary robotics strategy (i.e., where performance is measured via simulation), or with online learning.

We investigate the capability of a transformer pretrained on natural language to generalize to other modalities with minimal finetuning -- in particular, without finetuning of the self-attention and feedforward layers of the residual blocks. We consider such a model, which we call a Frozen Pretrained Transformer (FPT), and study finetuning it on a variety of sequence classification tasks spanning numerical computation, vision, and protein fold prediction. In contrast to prior works which investigate finetuning on the same modality as the pretraining dataset, we show that pretraining on natural language improves performance and compute efficiency on non-language downstream tasks. In particular, we find that such pretraining enables FPT to generalize in zero-shot to these modalities, matching the performance of a transformer fully trained on these tasks.

Monte Carlo Tree Search (MCTS) is a decision-making algorithm that consists in searching large combinatorial spaces represented by trees. In such trees, nodes denote states, also referred to as configurations of the problem, whereas edges denote transitions (actions) from one state to another. MCTS has been originally proposed in the work by Kocsis and Szepesvári (2006) and by Coulom (2006), as an algorithm for making computer players in Go. It was quickly called a major breakthrough (Gelly et al., 2012) as it allowed for a leap from 14 kyu, which is an average amateur level, to 5 dan, which is considered an advanced level but not professional yet. Before MCTS, bots for combinatorial games had been using various modifications of the min-max alpha-beta pruning algorithm (Junghanns, 1998) such as MTD(f) (Plaat, 2014) and hand-crafted heuristics. In contrast to them, MCTS algorithm is at its core aheuristic, which means that no additional knowledge is required other than just rules of a game (or a problem, generally speaking). However, it is possible to take advantage of heuristics and include them in the MCTS approach to make it more efficient and improve its convergence. Moreover, the given problem often tends to be so complex, from the combinatorial point of view, that some form of external help, e.g.

CCC, an AI Leader in Insurance Technology, Reports a More Than 50 Percent Year-Over-Year Increase in the Application of Deep Learning for Claims Processing CCC Information Services (CCC), an AI leader in insurance technology, reports a more than 50 percent year-over-year increase in claims processed using deep learning. The company also reports that more than 5 million unique claims have been processed using a CCC deep learning AI solution, and the number of claims using two or more of its AI applications has more than doubled year over year. Today, more than 75 auto insurers are actively applying CCC's advanced AI to power claims decisions and improve policyholder experiences. "The growth we're seeing in AI adoption is tied to our proven ability to deliver actionable insights with advancing precision," said John Goodson, senior vice president, technology, CCC. "The CCC Cloud platform is built for speed and scale, and today processes 2.7 billion transactions per day. We are powering AI innovations at scale and are proud to help each client use AI to provide a tailored experience for their customers."

Skin cancer is one of the most deadly cancers worldwide. Yet, it can be reduced by early detection. Recent deep-learning methods have shown a dermatologist-level performance in skin cancer classification. Yet, this success demands a large amount of centralized data, which is oftentimes not available. Federated learning has been recently introduced to train machine learning models in a privacy-preserved distributed fashion demanding annotated data at the clients, which is usually expensive and not available, especially in the medical field.

On May 2020, OpenAI introduced GPT-3 which is the third iteration of GPT language generation model series. The model boasted of a capacity of 175 billion parameters, more than 10 times than any other language model created before it. And to say that the model was a significant improvement would be an understatement. It could write essays on any topic without any inconsistencies in the output text. When trained on code samples, it could generate small code snippets (which were actually functional!) by getting a description of the task in English by the user. A group of developers trained it to update financial statements on Microsoft Excel based on casual description of transactions.

Apply recent advancements like BERT, OpenAI GPT-2, and T5 transformers to solve real-world problems in edtech.

Scientists from NVIDIA and Harvard have made a huge breakthrough in genetic research. They developed a deep-learning toolkit that is able to significantly cut down the time and cost needed to run rare and single-cell experiments. According to a study published in Nature Communications, the AtacWorks toolkit can run inference on a whole genome, a process that normally takes a little over two days, in just half an hour. It's able to do so thanks to NVIDIA's Tensor Core GPUs. AtacWorks works with ATAC-seq, a well-established method designed to find open areas in the genome of healthy and diseased cells. These "open areas" are subsections of a person's DNA that are used to determine and activate specific functions (think liver, blood or skin cells).

Neural networks are a series of algorithms that mimic the operations of a human brain to recognize relationships between vast amounts of data. They are used in a variety of applications in financial services, from forecasting and marketing research to fraud detection and risk assessment. A neural network has many layers. Each layer performs a specific function, and the complex the network is, the more the layers are. That's why a neural network is also called a multi-layer perceptron.