This paper describes the process of building an annotated corpus and training models for classical French literature, with a focus on theatre, and particularly comedies in verse. It was originally developed as a preliminary step to the stylometric analyses presented in Cafiero and Camps [2019]. The use of a recent lemmatiser based on neural networks and a CRF tagger allows to achieve accuracies beyond the current state-of-the art on the in-domain test, and proves to be robust during out-of-domain tests, i.e. up to 20th c. novels. I INTRODUCTION If many lemmatisers and POS taggers have been trained, and sometimes conceived, for French (e.g. Tellier et al. [2012], Urieli [2013]...), they usually focus on contemporary French and tools for Ancien Régime French remain scarce. One important exception is the TreeTagger [Schmid, 1995] model developed by Diwersy et al. [2017] for the Presto project [Vigier and Blumenthal, 2013-2017].

This article focuses on different aspects of pedestrian (crowd) modeling and simulation. The review includes: various modeling criteria, such as granularity, techniques, and factors involved in modeling pedestrian behavior, and different pedestrian simulation methods with a more detailed look at two approaches for simulating pedestrian behavior in traffic scenes. At the end, benefits and drawbacks of different simulation techniques are discussed and recommendations are made for future research. Pedestrian behavior in the context of driving can be modeled at the individual level, e.g. a pedestrian is intending to cross the road in a residential area, or as part of a crowd of individuals, e.g. a group of pedestrians is about to cross a signalized intersection. Note that the term crowd is often used in phenomenology where the behavior of large groups of people is studied in the context of municipal road design, building evacuation system planning, shopping mall structure design, and many similar applications. In traffic context, the term "pedestrian group" or group for short is often used. However, since a large body of the literature on pedestrian simulation is involving phenomenology, I use the term crowd throughout this report. Before discussing different methods of modeling pedestrian and crowd behavior, it is important to understand what constitutes a crowd.

Spell correction is used to detect and correct orthographic mistakes in texts. Most of the time, traditional dictionary lookup with string similarity methods is suitable for the languages that have a less complex structure such as the English language. However, the Azerbaijani language has a more complex structure and due to its morphological structure, the derivation of words is plenty that several words are derived from adding suffices, affixes to the words. Therefore, in this paper sequence to sequence model with an attention mechanism is used to develop spelling correction for Azerbaijani. Total 12000 wrong and correct sentence pairs used for training, and the model is tested on 1000 real-world misspelled words and F1-score results are 75% for distance 0, 90% for distance 1, and 96% for distance 2.

While deep neural networks show great performance on fitting to the training distribution, improving the networks' generalization performance to the test distribution and robustness to the sensitivity to input perturbations still remain as a challenge. Although a number of mixup based augmentation strategies have been proposed to partially address them, it remains unclear as to how to best utilize the supervisory signal within each input data for mixup from the optimization perspective. We propose a new perspective on batch mixup and formulate the optimal construction of a batch of mixup data maximizing the data saliency measure of each individual mixup data and encouraging the supermodular diversity among the constructed mixup data. This leads to a novel discrete optimization problem minimizing the difference between submodular functions. We also propose an efficient modular approximation based iterative submodular minimization algorithm for efficient mixup computation per each minibatch suitable for minibatch based neural network training. Our experiments show the proposed method achieves the state of the art generalization, calibration, and weakly supervised localization results compared to other mixup methods.

Genomic data provides researchers with an invaluable source of information to advance progress in biomedical research, personalized medicine, and drug development. At the same time, however, this data is extremely sensitive, which makes data sharing, and consequently availability, problematic if not outright impossible. As a result, organizations have begun to experiment with sharing synthetic data, which should mirror the real data's salient characteristics, without exposing it. In this paper, we provide the first evaluation of the utility and the privacy protection of five state-of-the-art models for generating synthetic genomic data. First, we assess the performance of the synthetic data on a number of common tasks, such as allele and population statistics as well as linkage disequilibrium and principal component analysis. Then, we study the susceptibility of the data to membership inference attacks, i.e., inferring whether a target record was part of the data used to train the model producing the synthetic dataset. Overall, there is no single approach for generating synthetic genomic data that performs well across the board. We show how the size and the nature of the training dataset matter, especially in the case of generative models. While some combinations of datasets and models produce synthetic data with distributions close to the real data, there often are target data points that are vulnerable to membership inference. Our measurement framework can be used by practitioners to assess the risks of deploying synthetic genomic data in the wild, and will serve as a benchmark tool for researchers and practitioners in the future.

Recognizing human actions based on videos has became one of the most popular areas of research in computer vision in recent years. This area has many applications such as surveillance, robotics, health care, video search and human-computer interaction. There are many problems associated with recognizing human actions in videos such as cluttered backgrounds, obstructions, viewpoints variation, execution speed and camera movement. A large number of methods have been proposed to solve the problems. This paper focus on spatial and temporal pattern recognition for the classification of videos using Deep Neural Networks. This model takes RGB images and Optical Flow as input data and outputs an action class number. The final recognition accuracy was about 94%.

Deep neural networks with batch normalization (BN-DNNs) are invariant to weight rescaling due to their normalization operations. However, using weight decay (WD) benefits these weight-scale-invariant networks, which is often attributed to an increase of the effective learning rate when the weight norms are decreased. In this paper, we demonstrate the insufficiency of the previous explanation and investigate the implicit biases of stochastic gradient descent (SGD) on BN-DNNs to provide a theoretical explanation for the efficacy of weight decay. We identity two implicit biases of SGD on BN-DNNs: 1) the weight norms in SGD training remain constant in the continuous-time domain and keep increasing in the discrete-time domain; 2) SGD optimizes weight vectors in fully-connected networks or convolution kernels in convolution neural networks by updating components lying in the input feature span, while leaving those components orthogonal to the input feature span unchanged. Thus, SGD without WD accumulates weight noise orthogonal to the input feature span, and cannot eliminate such noise. Our empirical studies corroborate the hypothesis that weight decay suppresses weight noise that is left untouched by SGD. Furthermore, we propose to use weight rescaling (WRS) instead of weight decay to achieve the same regularization effect, while avoiding performance degradation of WD on some momentum-based optimizers. Our empirical results on image recognition show that regardless of optimization methods and network architectures, training BN-DNNs using WRS achieves similar or better performance compared with using WD. We also show that training with WRS generalizes better compared to WD, on other computer vision tasks.

As gradient-free stochastic optimization gains emerging attention for a wide range of applications recently, the demand for uncertainty quantification of parameters obtained from such approaches arises. In this paper, we investigate the problem of statistical inference for model parameters based on gradient-free stochastic optimization methods that use only function values rather than gradients. We first present central limit theorem results for Polyak-Ruppert-averaging type gradient-free estimators. The asymptotic distribution reflects the trade-off between the rate of convergence and function query complexity. We next construct valid confidence intervals for model parameters through the estimation of the covariance matrix in a fully online fashion. We further give a general gradient-free framework for covariance estimation and analyze the role of function query complexity in the convergence rate of the covariance estimator. This provides a one-pass computationally efficient procedure for simultaneously obtaining an estimator of model parameters and conducting statistical inference. Finally, we provide numerical experiments to verify our theoretical results and illustrate some extensions of our method for various machine learning and deep learning applications.

Data acquisition forms the primary step in all empirical research. The availability of data directly impacts the quality and extent of conclusions and insights. In particular, larger and more detailed datasets provide convincing answers even to complex research questions. The main problem is that 'large and detailed' usually implies 'costly and difficult', especially when the data medium is paper and books. Human operators and manual transcription have been the traditional approach for collecting historical data. We instead advocate the use of modern machine learning techniques to automate the digitisation process. We give an overview of the potential for applying machine digitisation for data collection through two illustrative applications. The first demonstrates that unsupervised layout classification applied to raw scans of nurse journals can be used to construct a treatment indicator. Moreover, it allows an assessment of assignment compliance. The second application uses attention-based neural networks for handwritten text recognition in order to transcribe age and birth and death dates from a large collection of Danish death certificates. We describe each step in the digitisation pipeline and provide implementation insights.

The increasing and ubiquitous use of machine learning (ML) algorithms in many technological [1], financial [2, 3] and medical applications [4] calls for an improved understanding of their inner working that is, calls for more interpretable algorithms. Indeed difficulties in understanding how neural networks operate constitute a major problem in sensitive applications such as self-driving vehicles or medical diagnosis, where errors from the machine could result in otherwise avoidable accidents and human losses. Actually, the impossibility to fully grasp the decision process undertaken by the network not only prevents humans from being able to supervise such decision and eventually correct it, but also hinders our ability to use these algorithms to better understand the problem under scrutiny, and to inspire new improved methods and approaches for solving it. Thus, the development and deployment of interpretable neural networks could represent an important step to improve the user trust and consequently to foster the adoption of Artificial Intelligence systems in common, everyday tasks [5, 6].