Recent progress in the task of Grammatical Error Correction (GEC) has been driven by addressing data sparsity, both through new methods for generating large and noisy pretraining data and through the publication of small and higher-quality finetuning data in the BEA-2019 shared task. Building upon recent work in Neural Machine Translation (NMT), we make use of both kinds of data by deriving example-level scores on our large pretraining data based on a smaller, higher-quality dataset. In this work, we perform an empirical study to discover how to best incorporate delta-log-perplexity, a type of example scoring, into a training schedule for GEC. In doing so, we perform experiments that shed light on the function and applicability of delta-log-perplexity. Models trained on scored data achieve state-of-the-art results on common GEC test sets.

Random Fourier features enable researchers to build feature map to learn the spectral distribution of the underlying kernel. Current distribution-based methods follow a two-stage scheme: they first learn and optimize the feature map by solving the kernel alignment problem, then learn a linear classifier on the features. However, since the ideal kernel in kernel alignment problem is not necessarily optimal in classification tasks, the generalization performance of the random features learned in this two-stage manner can perhaps be further improved. To address this issue, we propose an end-to-end, one-stage kernel learning approach, called generative random Fourier features, which jointly learns the features and the classifier. A generative network is involved to implicitly learn and to sample from the distribution of the latent kernel. Random features are then built via the generative weights and followed by a linear classifier parameterized as a full-connected layer. We jointly train the generative network and the classifier by solving the empirical risk minimization problem for a one-stage solution. Straightly minimizing the loss between predictive and true labels brings better generalization performance. Besides, this end-to-end strategy allows us to increase the depth of features, resulting in multi-layer architecture and exhibiting strong linear-separable pattern. Empirical results demonstrate the superiority of our method in classification tasks over other two-stage kernel learning methods. Finally, we investigate the robustness of proposed method in defending adversarial attacks, which shows that the randomization and resampling mechanism associated with the learned distribution can alleviate the performance decrease brought by adversarial examples.

The advancements in stellar spectroscopy data acquisition have made it necessary to accomplish similar improvements in efficient data analysis techniques. Current automated methods for analyzing spectra are either (a) data-driven, which requires prior knowledge of stellar parameters and elemental abundances, or (b) based on theoretical synthetic models that are susceptible to the gap between theory and practice. In this study, we present a hybrid generative domain adaptation method that turns simulated stellar spectra into realistic spectra by applying unsupervised learning to large spectroscopic surveys. We apply our technique to the APOGEE H-band spectra at R=22,500 and the Kurucz synthetic models. As a proof of concept, two case studies are presented. The first of which is the calibration of synthetic data to become consistent with observations. To accomplish this, synthetic models are morphed into spectra that resemble observations, thereby reducing the gap between theory and observations. Fitting the observed spectra shows an improved average reduced $\chi_R^2$ from 1.97 to 1.22, along with a reduced mean residual from 0.16 to -0.01 in normalized flux. The second case study is the identification of the elemental source of missing spectral lines in the synthetic modelling. A mock dataset is used to show that absorption lines can be recovered when they are absent in one of the domains. This method can be applied to other fields, which use large data sets and are currently limited by modelling accuracy. The code used in this study is made publicly available on github.

The Benelux Office for Intellectual Property (BOIP) has partnered with Clarivate Plc, a global leader in providing trusted information and insights to accelerate the pace of innovation, to improve its trademark research services. Using AI-powered technology from Clarivate, BOIP has simplified the process of researching image trademarks for uniqueness and availability. BOIP joins innovative IP offices around the world like the EU Intellectual Property Office, IP Australia and the Intellectual Property Office of Singapore who have adopted image recognition (IR)1 and new technologies to deliver innovative and more accessible services to users. Technology has transformed trademark research, automating a previously time-consuming and manual task. Today, the ability to search and compare image trademarks is essential as 40% of trademarks worldwide contain an image component2.

ThoughtRiver, a London-based legaltech startup that's applying AI to speed up contract pre-screening, has announced a $10 million Series A round of funding led by Octopus Ventures. Existing seed investors Crane, Local Globe, Entrée Capital, Syndicate Room, and angel investor Duncan Painter also participated in the round. The UK startup is one of a number applying AI to automate work that would otherwise be done by legal professions with the aim of boosting operational efficiency. Other startups playing in the space include the likes of Kira Systems, LawGeex and Luminance to name a few. ThoughtRiver argues it has a different focus vs the majority of contract view companies because it's focusing on pre-signature contracts -- with the aim of making securing a deal faster. "Almost all others are just employed to pull data from existing contracts.

Artificial intelligence (AI) in health care has arrived, with enormous potential for change in the delivery of care, but experts published in the Medical Journal of Australia today are asking if we are ready. "AI, machine learning, and deep neural network tools can assist medical decision making and management, and have already permeated into at least three different levels: AI-assisted image interpretation; AI-assisted diagnosis; and, AI-assisted prediction and prognostication," wrote the authors, Joseph Sung, the Mok Hing Yiu Professor of Medicine at the Chinese University of Hong Kong, Cameron Stewart, Professor of Health, Law and Ethics at the University of Sydney, and Professor Ben Freedman, the Deputy Director of Research Strategy at the Heart Research Institute and the University of Sydney's Charles Perkins Center and Concord Clinical School. "From diagnosing retinopathy to cardiac arrhythmias, from screening for skin cancer to breast cancer, from predicting outcome of stroke to self-management of chronic diseases, AI and machine learning devices can replace many time-consuming, labor-intensive, repetitive and mundane tasks of clinicians and give possible suggestions of management plans," Sung and colleagues wrote. The quality of AI in health care is dependent on the quality of the data on which it is based. "Algorithms are being developed and validated on data generated by health care systems where current practices may already be inequitable," they wrote.

AI contract review business ThoughtRiver has secured a $10m (£7.5m) investment from venture capitalists to expand its presence around the world, with extra spending in the US and South-East Asia. Octopus Ventures, which has £1.3bn under management and invests in technology innovation start-ups, pledged the Series A funding, saying it liked the company because it was disrupting a legal market that had been "slow to adopt AI" compared to other industries. ThoughtRiver was originally incubated by Cambridge law firm Taylor Vinters, which remains a shareholder in the business, and it is run by former Taylor Vinters partner Tim Pullan. The company, which has offices in Cambridge, London, New York, Singapore and New Zealand, has developed its own natural language processing engine that Mr Pullan said helped to iron out differences in the language used by lawyers to draft contracts. The product, Lexible, is used by PwC and businesses in the telecoms, retailing and technology sectors to automate the reviewing and pre-screening of contracts.

Simulators that can rapidly test trillions of options would accelerate the slow and costly process of human clinical trials. The magnitude of the Covid-19 pandemic will largely depend on how quickly safe and effective vaccines and treatments can be developed and tested. Many assume a widely available vaccine is years away, if ever. Others believe that a 12- to 18-month development cycle is a given. Our best bet to reduce even that record-breaking timeline is by using artificial intelligence. The problem is twofold: discovering the right set of molecules among billions of possibilities, and then waiting for clinical trials.

Learning from real-life complex networks is a lively research area, with recent advances in learning information-rich, low-dimensional network node representations. However, state-of-the-art methods offer little insights as the features that constitute the learned node representations are not interpretable and are as such less applicable to sensitive settings in biomedical or user profiling tasks, where bias detection is highly relevant. The proposed SNoRe (Symbolic Node Representations) algorithm is capable of learning symbolic, human-understandable representations of individual network nodes based on the similarity of neighborhood hashes to nodes chosen as features. SNoRe's interpretable features are suitable for direct explanation of individual predictions, which we demonstrate by coupling it with the widely used instance explanation tool SHAP to obtain nomograms representing the relevance of individual features for a given classification, which is to our knowledge one of the first such attempts in a structural node embedding setting. In the experimental evaluation on 11 real-life datasets, SNoRe proved to be competitive to strong baselines, such as variational graph autoencoders, node2vec and LINE. The vectorized implementation of SNoRe scales to large networks, making it suitable for many contemporary network analysis tasks.

Industry 4.0 will make manufacturing processes smarter but this smartness requires more environmental awareness, which in case of Industrial Internet of Things, is realized by the help of sensors. This article is about industrial pharmaceutical systems and more specifically, water purification systems. Purified water which has certain conductivity is an important ingredient in many pharmaceutical products. Almost every pharmaceutical company has a water purifying unit as a part of its interdependent systems. Early detection of faults right at the edge can significantly decrease maintenance costs and improve safety and output quality, and as a result, lead to the production of better medicines. In this paper, with the help of a few sensors and data mining approaches, an anomaly detection system is built for CHRIST Osmotron water purifier. This is a practical research with real-world data collected from SinaDarou Labs Co. Data collection was done by using six sensors over two-week intervals before and after system overhaul. This gave us normal and faulty operation samples. Given the data, we propose two anomaly detection approaches to build up our edge fault detection system. The first approach is based on supervised learning and data mining e.g. by support vector machines. However, since we cannot collect all possible faults data, an anomaly detection approach is proposed based on normal system identification which models the system components by artificial neural networks. Extensive experiments are conducted with the dataset generated in this study to show the accuracy of the data-driven and model-based anomaly detection methods.